Skip to main content
    Kriv AI

    Banking & Financial Services Governance

    How Banks Validate an AI Model Risk Management Consulting Vendor

    What a bank's third-party risk and model risk teams should check before engaging an AI model risk management consulting vendor.

    A bank can evaluate an AI model risk management vendor against two reference points: SR 26-2, the interagency model risk management guidance whose principles the vendor's methodology should be able to apply, and SR 23-4, the interagency guidance on third-party relationships, which describes due diligence and ongoing oversight for vendors a bank engages.

    context

    Two Reference Points a Bank Can Use

    Two pieces of interagency guidance are useful reference points for how a bank can engage an AI model risk management consulting vendor: one covers the vendor relationship itself, the other describes model risk management principles a vendor's methodology should be able to work with.

    SR 23-4: the third-party relationship itself

    The Federal Reserve, FDIC, and OCC issued interagency guidance on third-party relationships in June 2023 (SR 23-4 for Fed-supervised institutions), replacing each agency's separate prior guidance with one set of guidance. It describes a risk management life cycle for third-party relationships, including a consulting vendor: planning, due diligence and vendor selection, contract negotiation, ongoing monitoring, and termination. For an AI model risk management vendor, the due-diligence stage typically covers the vendor's financial condition, legal and regulatory history, business experience, and its technical competence to do model risk work in the first place.

    SR 26-2: the vendor's model risk methodology

    SR 26-2, the April 2026 interagency guidance that supersedes SR 11-7, describes model risk management principles a vendor should be able to work with, including validation of conceptual soundness, ongoing monitoring, and outcomes analysis, together with effective challenge. If SR 11-7 is still the term your team uses, our explainer on what SR 11-7 is and how it applies to AI models shows where it stands today. A bank can reasonably ask a vendor engaged to do model risk work on its AI systems how it would apply those principles to models that do not behave deterministically; SR 26-2 itself says generative AI and agentic AI models are not within its scope.

    ai gap

    Why Generic Model Risk Vendors Often Fall Short on AI

    Many consulting vendors built their model risk practice around traditional statistical and econometric models: credit scoring, PD/LGD, stress testing. Those models are deterministic, so validating them is a matter of testing developer documentation, replicating outputs, and checking assumptions.

    AI and machine learning models, especially generative and agentic systems, break that assumption: the same input can produce different output across runs, and the reasoning behind an output is often not directly inspectable. A vendor whose methodology has no answer for that, no structured output sampling, no drift monitoring, no plan for testing non-deterministic behavior, may have no way to monitor and test an AI system the way SR 26-2 describes for traditional models, regardless of how strong its traditional model risk practice is.

    checklist

    Eight Questions to Ask an AI Model Risk Management Vendor

    1. 1. Can you validate non-deterministic models?

      The vendor should describe a specific method, such as structured output sampling against a scoring rubric and drift monitoring over time, not just traditional back-testing built for deterministic models.

    2. 2. Is the validation team independent of any AI system the vendor also helped build?

      SR 26-2 says effective challenge needs sufficient independence to maintain objectivity. If the same vendor built and validates a bank's AI model, a bank should consider that conflict and how to manage or avoid it.

    3. 3. What does the deliverable actually contain?

      A validation package should include the risk assessment, the evidence gathered, the validation protocol, and an ongoing monitoring plan, not a one-time slide deck.

    4. 4. How does the vendor handle model drift after the initial validation?

      SR 26-2 treats ongoing monitoring as part of sound model risk management alongside initial validation. Ask what cadence and triggers the vendor uses to flag drift after go-live.

    5. 5. What is the vendor's own financial condition and business continuity plan?

      SR 23-4 due diligence covers the vendor's financial condition and its ability to keep operating, not just its technical work.

    6. 6. How does the vendor handle its own subcontractors or fourth parties?

      If the vendor relies on subcontractors or other outside tools to do the work, SR 23-4 describes evaluating the vendor's reliance on subcontractors, so the bank needs to know who they are.

    7. 7. What does exit look like if the bank needs to switch vendors?

      SR 23-4 describes planning for termination. Ask how the vendor transitions documentation, model inventories, and validation history to the bank or a successor vendor.

    8. 8. Can the vendor point to specific model risk management experience, not just AI experience?

      General AI consulting experience is not the same as model risk management experience. Ask which specific frameworks, such as SR 26-2 or OCC guidance, the vendor's staff have actually worked under.

    plain answer

    AI model validation vendors: what to compare

    AI model validation vendors can look alike on a proposal, so compare them on four things. First, method: can the vendor test models whose output changes from run to run, for example with structured output sampling against a scoring rubric and drift monitoring, or only with back-testing built for deterministic models? Second, independence: did the vendor help build the model it is validating? SR 26-2 says effective challenge needs "sufficient independence to maintain objectivity". Third, the deliverable: a risk assessment, the evidence, the validation protocol and a monitoring plan. Fourth, exit: how would the documentation and validation history move to the bank or a successor vendor? The eight questions above cover each of these in more detail.

    plain answer

    Who can validate vendor AI models under SR 26-2?

    Under SR 26-2, validation of vendor products can be done "either by internal or outside parties". The guidance's section on vendor and other third-party products calls that validation an important element of model risk management. It says sound practice includes understanding the vendor model's conceptual soundness, design, development data and performance, plus ongoing monitoring and outcome analysis. Two limits apply. First, SR 26-2 says generative AI and agentic AI models are not within its scope, so for those a bank's own risk management and governance practices should guide the controls. Second, the guidance is written for banking organizations, and your supervisor's expectations for your institution still apply. Read on 11 November 2026 from the Federal Reserve's PDF.

    engagement

    How Kriv AI Fits Into This Process

    Kriv AI works with banks on this in two ways: as the AI model risk management vendor being evaluated against a checklist like the one above, and as an independent second-opinion reviewer brought in to validate another vendor's work or a bank's own internal AI model risk program before an exam.

    Either way, the engagement starts with the same inventory-first approach used for SR 26-2 work generally: every AI or ML system in scope, mapped to its risk tier, with a validation plan that accounts for non-deterministic behavior rather than assuming a traditional deterministic test script will do the job.

    tiers

    What You Get at Each Tier

    1. 1. Enterprise / regulated (banks, financial services)

      A full AI model risk validation program under SR 26-2, or an independent second-opinion review of an existing vendor's or internal team's validation work, with a documentation package ready for exam.

    2. 2. Fractional AI governance lead

      Ongoing oversight of vendor relationships and model validation work as new AI systems enter the pipeline, plus regular reporting to model risk and compliance leadership.

    3. 3. Specialized advisory

      A targeted review of one AI system's validation package, or a second opinion ahead of a regulatory exam or vendor renewal decision.

    engagement

    How engagements are scoped and priced

    Every engagement is scoped and priced after a short discovery call.

    Straight answers

    Frequently asked questions about How Banks Validate an AI Model Risk Management Consulting Vendor

    What is SR 23-4?

    SR 23-4 is the Federal Reserve's version of the June 2023 interagency guidance on third-party relationships, issued jointly with the FDIC and OCC. It sets out a risk management life cycle, planning, due diligence, contract negotiation, ongoing monitoring, and termination, for third-party relationships, including an AI model risk management consulting vendor.

    What is SR 26-2?

    SR 26-2 is the April 2026 interagency guidance on model risk management from the Federal Reserve, OCC, and FDIC, superseding SR 11-7. It describes conceptual soundness review, ongoing monitoring, and outcomes analysis as parts of sound model risk management, together with effective challenge.

    Does an AI model risk vendor need to be independent from the model builder?

    SR 26-2 says effective challenge needs sufficient independence to maintain objectivity. If the same vendor both built and validates a bank's AI model, a bank should consider that conflict of interest and how to manage or avoid it.

    What should a bank ask before hiring an AI model risk management vendor?

    Whether the vendor can validate non-deterministic models, whether its validation team is independent, what the deliverable actually contains, how it monitors drift after go-live, and whether it can point to specific model risk management experience rather than general AI experience.

    What does AI model risk vendor validation cost?

    Every engagement is scoped and priced after a short discovery call.

    Does Kriv AI work with smaller community banks?

    Kriv AI's own consulting practice focuses on enterprise and regulated banking and financial services organizations.

    What should I compare across AI model validation vendors?

    Compare method for models whose output changes between runs, independence from the team that built the model, what the deliverable contains, and how documentation and validation history move to you at exit.

    Can an outside party validate a vendor model under SR 26-2?

    SR 26-2 says validation of vendor products can be done "either by internal or outside parties" (read 11 November 2026). It also says generative AI and agentic AI models are outside its scope.

    Talk to the team that would do the work

    Bring your requirements to a working session with the person who'll actually deliver.

    Book a Discovery Call