Vendor & Third-Party Risk
AI Vendor Evaluation Checklist for Healthcare Compliance
The questions a compliance team should ask before any AI vendor touches PHI, claims data, or a clinical workflow, and why a signed BAA is where the review starts, not where it ends.
Evaluating an AI vendor for healthcare compliance means checking five things before any contract is signed: how the vendor handles protected health information under HIPAA, whether it will sign a business associate agreement, how it validates and monitors its models, what happens to your data if you leave, and who is liable when the AI is wrong.
evaluate ai vendor
How to Evaluate an AI Vendor for Healthcare Compliance
Evaluating an AI vendor for healthcare compliance means checking five things before any contract is signed: how the vendor handles protected health information under HIPAA, whether it will sign a business associate agreement, how it validates and monitors its models, what happens to your data if you leave, and who is liable when the AI is wrong.
Most healthcare organizations already have a vendor security review process built around HIPAA and SOC 2. An AI vendor needs an additional layer on top of that, because the risk is not just where data sits, it is what a model does with it, whether that behavior changes without notice, and whether you can explain an output to an auditor or a patient months after the fact.
The checklist below is organized the way a compliance or clinical informatics team actually works through a vendor review: start with the legal and data-handling basics, move into model-specific due diligence, then into contract and exit terms that most ordinary software reviews skip entirely. Use it for a new AI feature request, a formal RFP, or a re-review of a vendor you already use whose product just added generative or agentic capability.
core domains healthcare
The Core Domains a Healthcare AI Vendor Checklist Should Cover
A complete evaluation spans six domains. Skipping any one of them is how organizations end up with a signed BAA and a vendor whose model behavior, data retention, or subprocessor list they still cannot describe accurately.
1. Data handling and the BAA
Confirm the vendor will sign a business associate agreement that names AI-specific uses of PHI, not just storage and transmission, including whether PHI is used to train or fine-tune models.
2. Model transparency and validation
Ask what the model actually is (proprietary, a fine-tuned open model, or a wrapper around a third-party API), how it was validated for your use case, and how performance is monitored after deployment.
3. Security and access controls
Review encryption at rest and in transit, role-based access, logging, and whether the vendor's own AI features, such as embedded copilots or chat, can see data outside the workflow you approved.
4. Subprocessors and the AI supply chain
Get the full list of downstream model providers and infrastructure vendors, because your BAA obligations extend to every subprocessor that touches PHI, including the underlying foundation model provider.
5. Contract terms and liability
Check who is responsible when a model output is wrong, what the vendor discloses about model changes, and whether pricing or feature access depends on data-sharing terms you have not agreed to.
6. Exit and portability
Confirm you can retrieve your data and any model artifacts trained on it in a usable format, and understand what breaks in downstream workflows if you switch vendors later.
building rfp put
Building the RFP: What to Put in Writing
A healthcare AI governance RFP should force written, attributable answers on model provenance, validation methodology, and PHI handling, not marketing language about accuracy or blanket HIPAA claims. Vague answers here are a reliable predictor of problems after signature.
| RFP section | What to ask | Why it matters |
|---|---|---|
| Model provenance | Which underlying model or models power this feature, and does that ever change without customer notice? | A vendor that swaps foundation models silently can change output behavior and your risk profile overnight. |
| Validation methodology | How was the model validated for this specific clinical or claims use case, and can you share the validation report? | Generic accuracy claims on a vendor's marketing page are not evidence of validation for your population or workflow. |
| PHI and training data | Is customer PHI used to train, fine-tune, or improve models, and can this be contractually excluded? | This is the most common gap between what a sales team says and what the data processing addendum actually permits. |
| Monitoring and drift | How is model output monitored in production, and what triggers a re-validation or rollback? | Without a monitoring answer, you are relying on the vendor to notice degradation before your clinicians or claims staff do. |
| Incident history | Has this product had a security incident, a model behavior incident, or a regulatory inquiry in recent years? | A vendor that has never had to answer this question in an RFP before is a signal, not a reassurance. |
due diligence health
Due Diligence for Health System AI Vendors Beyond the RFP Response
An RFP response is a claim. Due diligence is verifying it. For a health system, that means going past the written answers into references, audit rights, and the parts of the vendor relationship that only show up once something goes wrong.
Ask for at least one reference from an organization of comparable size and EHR environment, and ask that reference specifically about model behavior changes and support responsiveness, not just uptime.
Confirm the contract gives you audit rights, or at minimum a right to request a current SOC 2 Type II report and any available third-party model validation summary on a recurring basis, rather than once at signing.
Check whether the vendor has a documented incident response process for AI-specific failures, such as a model producing an unsafe clinical suggestion or a data leak through a chat interface, separate from its general security incident process.
Verify insurance coverage extends to AI-related errors and omissions, not only to standard data breach and cyber liability, since some cyber policies exclude harm caused by model output rather than a traditional breach.
finding hidden ai
Finding Hidden AI Exposure in Your Vendor Stack
Assessing AI vendor exposure in healthcare starts with an inventory, because most organizations have more AI in production than their vendor list shows: EHR, billing, and scheduling platforms have added generative and agentic features to existing contracts without a new sales conversation or a new BAA line item.
Start by pulling every vendor with system access to PHI and checking release notes and admin settings for AI features added recently, not just the vendors you signed specifically for AI.
Pay particular attention to features that are on by default, such as an AI-generated visit summary, an AI coding suggestion, or a chat assistant embedded in a portal, since these can process PHI through a new model path that predates your original security review.
Where a vendor has added AI to an existing product, treat it as a new due diligence event, not an automatic extension of the trust you already granted for the non-AI parts of that product.
ehr platform lock
EHR and Platform Lock-In: What to Check Before You Sign
AI vendor lock-in risk is highest with EHR-embedded AI, because the switching cost is not just the software, it is the clinical workflows, coded data structures, and staff training built around one vendor's model behavior. The checklist question is what happens to that investment if you need to leave.
Ask whether AI-generated content, such as summaries, coded suggestions, or extracted structured data, is stored in an open, exportable format or only accessible through the vendor's proprietary interface.
Check whether your clinical or claims workflows depend on model-specific behavior, prompt templates, or fine-tuning that would need to be rebuilt from scratch with a different vendor, and get an honest estimate of that rebuild effort before you need it.
Confirm pricing and feature access are not structured to penalize reduced usage or data export, since some AI add-ons are priced or gated in ways that make a phased exit more expensive than it should be.
Where the AI feature is bundled into a broader EHR or platform contract, negotiate the AI terms as a separable clause so a change in the AI feature does not force a renegotiation of the entire platform agreement.
adapting same checklist
Adapting the Same Checklist for Financial Services
The domains do not change much outside healthcare. A finserv AI vendor evaluation checklist swaps HIPAA and the BAA for the applicable financial privacy and model risk framework, but still runs through provenance, validation, monitoring, contract terms, and exit.
For a bank, credit union, or insurer, the model risk section of the checklist should map to your existing model risk management framework, most commonly built around the Federal Reserve's SR 11-7 guidance and its 2026 update, SR 26-2, rather than treating vendor AI review as a separate process from model validation you already run internally.
Insurers evaluating an AI vendor for underwriting, claims, or pricing should specifically ask how the vendor supports the documentation and testing expectations described in the NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, since a number of states have adopted it directly.
The NIST AI Risk Management Framework is a useful common vocabulary across both industries for structuring the questions in this checklist, even though it is voluntary rather than a regulatory mandate in either sector.
kriv ai runs
How Kriv AI Runs a Vendor Evaluation Engagement
Kriv AI runs vendor evaluations as a structured engagement rather than a one-time questionnaire review, because vendor AI capability changes faster than most annual security review cycles account for.
This is due diligence and governance advisory work, not a vendor certification and not a warranty of any vendor's compliance status. The output is a documented evaluation your team can defend to an auditor, a board, or a regulator, and a clear list of what to fix in the contract before you sign.
1. Inventory and scoping
Map every vendor with system access to PHI, claims data, or financial customer data, and flag which ones have added AI features since the last review.
2. Checklist and RFP support
Build or adapt the evaluation checklist and RFP questions to your regulatory environment, whether that is HIPAA and state health privacy law or SR 11-7 and SR 26-2 model risk expectations.
3. Vendor response review
Read vendor responses and supporting documents, including SOC 2 reports, validation summaries, and subprocessor lists, against the checklist and flag gaps in writing, not verbally.
4. Contract and exit terms
Work with your legal team on the AI-specific contract language: PHI training-data exclusions, audit rights, incident disclosure, and data portability on exit.
Sources
Cited sources
- SR 11-7, the Federal Reserve's 2011 guidance on model risk management
- SR 26-2, the Federal Reserve's 2026 revised guidance on model risk management
- 21 CFR Part 11 guidance on electronic records and electronic signatures
- NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers
- NIST AI Risk Management Framework
Straight answers
Frequently asked questions about AI Vendor Evaluation Checklist for Healthcare Compliance
What should a healthcare AI governance RFP checklist include?
A healthcare AI governance RFP should require written answers on model provenance (which model, and whether it changes without notice), validation methodology for your specific use case, whether PHI is used to train or fine-tune the model, how output is monitored in production, and the vendor's incident history. Marketing language about HIPAA readiness is not a substitute for these specifics.
What is a health system AI vendor due diligence checklist, and how is it different from an RFP?
The RFP captures what a vendor claims in writing. Due diligence is verifying it: checking references from comparably sized health systems, confirming audit rights and recurring access to SOC 2 and validation reports, reviewing the vendor's AI-specific incident response process, and confirming insurance covers AI-related errors, not just data breaches.
How do we assess AI vendor exposure across our existing systems, not just new AI purchases?
Start with an inventory of every vendor that already has system access to PHI, then check release notes and admin settings for AI features added to existing contracts recently. EHR, billing, and scheduling vendors frequently add generative or agentic features to products you already trust, without a new BAA line item or a new security review.
What is AI vendor lock-in risk with an EHR platform?
It is the risk that AI-generated content and the workflows built around it are only usable inside one vendor's interface, so leaving means rebuilding clinical templates, coded data extraction, and staff training from scratch. Check whether AI output is stored in an open format and whether the AI terms can be separated from the broader EHR contract before you sign.
Is a finserv AI vendor evaluation checklist different from a healthcare one?
The structure is the same: provenance, validation, monitoring, contract terms, and exit. The regulatory anchor changes: financial institutions map the model risk section to the Federal Reserve's SR 11-7 guidance and its SR 26-2 update, and insurers add the NAIC Model Bulletin on AI systems, in place of HIPAA and the BAA.
Does a signed business associate agreement make an AI vendor HIPAA-ready?
No. A BAA establishes the legal obligation to protect PHI, but it does not verify how the vendor's model actually behaves, whether PHI is used in training, or how the vendor monitors output. Treat the BAA as a required starting point, and run the rest of the checklist to verify the practices behind it.
How do I evaluate an AI vendor for healthcare compliance if we do not have an internal model risk team?
Use the same six domains a model risk team would apply: data handling and the BAA, model transparency and validation, security controls, the subprocessor list, contract and liability terms, and exit and portability. An outside governance advisor can run the RFP and due diligence process for you and hand your legal team the specific contract language to negotiate.
Talk to the team that would do the work
Bring your requirements to a working session with the person who'll actually deliver.
Book a Discovery Call