Why Is NAIC AI Model Governance So Hard to Implement for AI?

The NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers was adopted by NAIC membership in December 2023 as a single template. In practice it never stayed one template. By late 2025, 23 states and Washington, D.C. had adopted the bulletin in some form, and adoption has kept accelerating through 2026. The problem for a compliance team is that not every state adopted it verbatim. States including Colorado, Virginia, Connecticut, and Pennsylvania have moved toward more prescriptive requirements layered on top of the bulletin's baseline, and Colorado's own regulation under SB 21-169 already imposes separate testing and attestation obligations on life, auto, and health benefit insurers that go beyond what the bulletin itself requires.

For a carrier licensed in a dozen or more states, that means a single AI governance program has to satisfy a dozen or more slightly different sets of expectations at once, not one shared standard. A governance document written to the bulletin's general language can still leave a carrier short in a state that added its own documentation, testing, or attestation layer, and there is no single master checklist that covers every variant. That is the first reason implementation is hard: the target is not fixed, and it is not the same target in every state where the carrier operates.

Most insurers do not build their underwriting, claims, or fraud-detection AI in-house. They license it. That creates the second implementation problem: outsourcing the AI does not outsource the compliance obligation. The 2023 Model Bulletin already required carriers to apply the same governance rigor to third-party models as to internally built ones, and the NAIC's advancing proposal for a third-party AI vendor registry does not change that. Registration, where it exists, creates regulatory visibility into a vendor's model, not a safe harbor for the carrier that uses it. A carrier that uses a registered model is still answerable for the model's behavior.

In practice, this means a compliance team has to go back to every AI vendor and ask for training data sources, validation records, bias testing methodology, and known limitations, and many vendors simply cannot produce that documentation on request. When a vendor cannot provide model documentation, validation records, or audit support, that gap becomes the carrier's problem during an exam, not the vendor's. Reconstructing that vendor evidence retroactively, once an examiner has already asked for it, takes far longer and costs far more than building it into the vendor contract and onboarding process from the start.

The third reason implementation is hard is that the standard itself is not finished. The NAIC's AI Systems Evaluation Tool, the structured framework examiners will use to assess an insurer's AI governance during market conduct and financial exams, is currently in a multistate pilot running from March through September 2026 across 12 states: California, Colorado, Connecticut, Florida, Iowa, Louisiana, Maryland, Pennsylvania, Rhode Island, Vermont, Virginia, and Wisconsin. The NAIC plans to revise the Tool based on pilot feedback in September and October 2026, re-expose it for public comment, and consider it for adoption at the NAIC's Fall National Meeting in November 2026.

That means any governance program built today is being built against a standard that is still being field-tested by regulators themselves. A carrier that documents exactly what today's pilot version of the Tool asks for could still see the bar move once the revised version is adopted later in 2026. Building a program that is directionally right, covering governance structure, model inventory, validation, bias testing, third-party oversight, and consumer-impact assessment, matters more than optimizing for the exact wording of a tool that is still being revised.

The fourth reason is technical, not just procedural. The bulletin and the state laws layered on top of it expect quantitative bias testing, not a policy statement that the carrier cares about fairness. That means measuring disparate impact in model outputs across protected classes, checking calibration by subgroup, and testing for proxy variables such as zip code or credit attributes that can reproduce a prohibited bias even when the model never uses a protected characteristic directly. Building and running that testing pipeline, and keeping a written record of the data used, the metric values computed, and who reviewed the result, is a data science and compliance function most carriers have never had to build together before.

The gap is measurable. Despite 92% of health insurers reporting current or planned AI use, nearly one-third of health insurers still do not regularly test their models for bias or discrimination. That gap between AI adoption and AI testing is exactly what an examiner is now trained to look for, and it is why so many carriers that believe they are compliant are not, once someone actually asks for the test results.