Banking & Financial Services Governance
How Banks Validate an AI Model Risk Management Consulting Vendor
What a bank's third-party risk and model risk teams should check before engaging an AI model risk management consulting vendor.
A bank can evaluate an AI model risk management vendor against two reference points: SR 26-2, the interagency model risk management guidance whose principles the vendor's methodology should be able to apply, and SR 23-4, the interagency guidance on third-party relationships, which describes due diligence and ongoing oversight for vendors a bank engages.
context
Two Reference Points a Bank Can Use
Two pieces of interagency guidance are useful reference points for how a bank can engage an AI model risk management consulting vendor: one covers the vendor relationship itself, the other describes model risk management principles a vendor's methodology should be able to work with.
SR 23-4: the third-party relationship itself
The Federal Reserve, FDIC, and OCC issued interagency guidance on third-party relationships in June 2023 (SR 23-4 for Fed-supervised institutions), replacing each agency's separate prior guidance with one set of guidance. It describes a risk management life cycle for third-party relationships, including a consulting vendor: planning, due diligence and vendor selection, contract negotiation, ongoing monitoring, and termination. For an AI model risk management vendor, the due-diligence stage typically covers the vendor's financial condition, legal and regulatory history, business experience, and its technical competence to do model risk work in the first place.
SR 26-2: the vendor's model risk methodology
SR 26-2, the April 2026 interagency guidance that supersedes SR 11-7, describes model risk management principles a vendor should be able to work with, including validation of conceptual soundness, ongoing monitoring, and outcomes analysis, together with effective challenge. If SR 11-7 is still the term your team uses, our explainer on what SR 11-7 is and how it applies to AI models shows where it stands today. A bank can reasonably ask a vendor engaged to do model risk work on its AI systems how it would apply those principles to models that do not behave deterministically; SR 26-2 itself says generative AI and agentic AI models are not within its scope.
ai gap
Why Generic Model Risk Vendors Often Fall Short on AI
Many consulting vendors built their model risk practice around traditional statistical and econometric models: credit scoring, PD/LGD, stress testing. Those models are deterministic, so validating them is a matter of testing developer documentation, replicating outputs, and checking assumptions.
AI and machine learning models, especially generative and agentic systems, break that assumption: the same input can produce different output across runs, and the reasoning behind an output is often not directly inspectable. A vendor whose methodology has no answer for that, no structured output sampling, no drift monitoring, no plan for testing non-deterministic behavior, may have no way to monitor and test an AI system the way SR 26-2 describes for traditional models, regardless of how strong its traditional model risk practice is.
checklist
Eight Questions to Ask an AI Model Risk Management Vendor
1. Can you validate non-deterministic models?
The vendor should describe a specific method, such as structured output sampling against a scoring rubric and drift monitoring over time, not just traditional back-testing built for deterministic models.
2. Is the validation team independent of any AI system the vendor also helped build?
SR 26-2 says effective challenge needs sufficient independence to maintain objectivity. If the same vendor built and validates a bank's AI model, a bank should consider that conflict and how to manage or avoid it.
3. What does the deliverable actually contain?
A validation package should include the risk assessment, the evidence gathered, the validation protocol, and an ongoing monitoring plan, not a one-time slide deck.
4. How does the vendor handle model drift after the initial validation?
SR 26-2 treats ongoing monitoring as part of sound model risk management alongside initial validation. Ask what cadence and triggers the vendor uses to flag drift after go-live.
5. What is the vendor's own financial condition and business continuity plan?
SR 23-4 due diligence covers the vendor's financial condition and its ability to keep operating, not just its technical work.
6. How does the vendor handle its own subcontractors or fourth parties?
If the vendor relies on subcontractors or other outside tools to do the work, SR 23-4 describes evaluating the vendor's reliance on subcontractors, so the bank needs to know who they are.
7. What does exit look like if the bank needs to switch vendors?
SR 23-4 describes planning for termination. Ask how the vendor transitions documentation, model inventories, and validation history to the bank or a successor vendor.
8. Can the vendor point to specific model risk management experience, not just AI experience?
General AI consulting experience is not the same as model risk management experience. Ask which specific frameworks, such as SR 26-2 or OCC guidance, the vendor's staff have actually worked under.
plain answer
AI model validation vendors: what to compare
AI model validation vendors can look alike on a proposal, so compare them on four things. First, method: can the vendor test models whose output changes from run to run, for example with structured output sampling against a scoring rubric and drift monitoring, or only with back-testing built for deterministic models? Second, independence: did the vendor help build the model it is validating? SR 26-2 says effective challenge needs "sufficient independence to maintain objectivity". Third, the deliverable: a risk assessment, the evidence, the validation protocol and a monitoring plan. Fourth, exit: how would the documentation and validation history move to the bank or a successor vendor? The eight questions above cover each of these in more detail.
plain answer
Who can validate vendor AI models under SR 26-2?
Under SR 26-2, validation of vendor products can be done "either by internal or outside parties". The guidance's section on vendor and other third-party products calls that validation an important element of model risk management. It says sound practice includes understanding the vendor model's conceptual soundness, design, development data and performance, plus ongoing monitoring and outcome analysis. Two limits apply. First, SR 26-2 says generative AI and agentic AI models are not within its scope, so for those a bank's own risk management and governance practices should guide the controls. Second, the guidance is written for banking organizations, and your supervisor's expectations for your institution still apply. Read on 11 November 2026 from the Federal Reserve's PDF.
engagement
How Kriv AI Fits Into This Process
Kriv AI works with banks on this in two ways: as the AI model risk management vendor being evaluated against a checklist like the one above, and as an independent second-opinion reviewer brought in to validate another vendor's work or a bank's own internal AI model risk program before an exam.
Either way, the engagement starts with the same inventory-first approach used for SR 26-2 work generally: every AI or ML system in scope, mapped to its risk tier, with a validation plan that accounts for non-deterministic behavior rather than assuming a traditional deterministic test script will do the job.
tiers
What You Get at Each Tier
1. Enterprise / regulated (banks, financial services)
A full AI model risk validation program under SR 26-2, or an independent second-opinion review of an existing vendor's or internal team's validation work, with a documentation package ready for exam.
2. Fractional AI governance lead
Ongoing oversight of vendor relationships and model validation work as new AI systems enter the pipeline, plus regular reporting to model risk and compliance leadership.
3. Specialized advisory
A targeted review of one AI system's validation package, or a second opinion ahead of a regulatory exam or vendor renewal decision.
engagement
How engagements are scoped and priced
Every engagement is scoped and priced after a short discovery call.
Sources
Cited sources
- Federal Reserve SR 23-4, Interagency Guidance on Third-Party Relationships: Risk Management (June 2023)
- Federal Reserve SR 26-2, Revised Guidance on Model Risk Management (April 17, 2026, supersedes SR 11-7)
- OCC Bulletin 2023-17, Third-Party Relationships: Interagency Guidance on Risk Management
- Federal Reserve, Third-Party Risk Management: A Guide for Community Banks
- Federal Reserve, SR 26-2 attachment (Revised Guidance on Model Risk Management, April 17, 2026), footnote 3, effective challenge paragraph and section VII
Related resources
Continue exploring
Straight answers
Frequently asked questions about How Banks Validate an AI Model Risk Management Consulting Vendor
What is SR 23-4?
SR 23-4 is the Federal Reserve's version of the June 2023 interagency guidance on third-party relationships, issued jointly with the FDIC and OCC. It sets out a risk management life cycle, planning, due diligence, contract negotiation, ongoing monitoring, and termination, for third-party relationships, including an AI model risk management consulting vendor.
What is SR 26-2?
SR 26-2 is the April 2026 interagency guidance on model risk management from the Federal Reserve, OCC, and FDIC, superseding SR 11-7. It describes conceptual soundness review, ongoing monitoring, and outcomes analysis as parts of sound model risk management, together with effective challenge.
Does an AI model risk vendor need to be independent from the model builder?
SR 26-2 says effective challenge needs sufficient independence to maintain objectivity. If the same vendor both built and validates a bank's AI model, a bank should consider that conflict of interest and how to manage or avoid it.
What should a bank ask before hiring an AI model risk management vendor?
Whether the vendor can validate non-deterministic models, whether its validation team is independent, what the deliverable actually contains, how it monitors drift after go-live, and whether it can point to specific model risk management experience rather than general AI experience.
What does AI model risk vendor validation cost?
Every engagement is scoped and priced after a short discovery call.
Does Kriv AI work with smaller community banks?
Kriv AI's own consulting practice focuses on enterprise and regulated banking and financial services organizations.
What should I compare across AI model validation vendors?
Compare method for models whose output changes between runs, independence from the team that built the model, what the deliverable contains, and how documentation and validation history move to you at exit.
Can an outside party validate a vendor model under SR 26-2?
SR 26-2 says validation of vendor products can be done "either by internal or outside parties" (read 11 November 2026). It also says generative AI and agentic AI models are outside its scope.
Talk to the team that would do the work
Bring your requirements to a working session with the person who'll actually deliver.
Book a Discovery Call