Why Did Our AI Claims Model Get Flagged by State Regulators?

The NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, adopted in December 2023, is the baseline nearly every state examiner now uses. It requires insurers to adopt, implement, and maintain a written AI governance program, called an AIS Program, that covers the full insurance lifecycle: underwriting, rating, marketing, fraud detection, and claim administration and payment. As of early 2026, 24 states have adopted the bulletin. If your claims model exists but your governance documentation does not, an examiner has grounds to flag the gap on its own, regardless of whether the model's outputs are actually biased.

Examiners are not asking whether you used AI. They are asking whether you can produce a governance file: who owns the model, what data feeds it, how it was validated before deployment, and who signs off on changes. A claims model with no named accountable owner, no change log, and no board or senior management sign-off is a documentation failure before it is ever tested for bias.

A model can exclude race, sex, and other protected characteristics from its inputs entirely and still produce outcomes that correlate with them, because zip code, prior claim history, and repair-cost estimators can act as proxies. Colorado's SB 21-169 and its implementing regulation require life, auto, and health benefit insurers to run quantitative testing on algorithms and predictive models built on external consumer data, and to document the results. New York's Circular Letter 7, adopted July 11, 2024, goes further for underwriting and pricing models: insurers must assess for disproportionate adverse effects on protected classes, evaluate legitimate explanations for any differential effect, and search for less discriminatory alternatives, before deployment and at least annually after.

A claims severity or fraud-flagging model that delays or downgrades payouts more often for policyholders in specific zip codes or demographic groups creates exactly this kind of adverse-impact exposure. Regulators do not need to prove intent. A pattern in the outcomes is enough to open an inquiry.

New York's guidance requires that any external data or AI system used in a covered decision have a clear, empirical, statistically significant, and rational relationship to the risk or outcome it is used to assess, and that the relationship not be unfairly discriminatory. In a claims context, that means a denial, a reduced payout, or a fraud referral produced or influenced by a model needs a defensible, documented reason, not just a score. If a claims adjuster cannot explain why the model flagged a claim, and the file contains no record of the model's validation or its limits, that is what a market conduct examiner writes up.

This is also why vendor-built claims tools carry regulatory risk even when the insurer did not build the model. The NAIC bulletin explicitly extends AIS Program obligations to third-party AI, so an insurer cannot point to a vendor's black box as a reason it cannot produce documentation.

AI claims tools are increasingly the direct subject of investigations, not just a side issue. In November 2025, Los Angeles County opened a civil investigation into State Farm's handling of wildfire claims from the Eaton and Palisades fires, and its formal document request specifically sought records reflecting the insurer's use of AI tools in the review of claims. Separately, a California Department of Insurance market conduct examination of the same claims found 398 violations of state law across a sample of 220 claims, including failures to investigate, accept, or deny claims within required timeframes and unreasonably low settlement offers.

That examination was not framed around AI specifically, but it illustrates the standard examiners apply to any claims process, human or automated: can the insurer show it followed a documented, defensible process for every claim, and can it produce that documentation on request. An AI claims model that cannot produce equivalent records is held to the same bar and fails it faster.