Skip to main content
    Kriv AI

    Healthcare AI Governance

    AI HIPAA compliance: what HIPAA requires before AI touches PHI

    HIPAA has no separate rulebook for artificial intelligence and no product certification. The same Privacy, Security and Breach Notification Rules apply to an AI tool as to any other system that handles protected health information. This page sets out what those rules require, cited to the regulation, and where they stop.

    AI HIPAA compliance is not a feature a tool ships with. HIPAA binds covered entities and business associates, so an AI system handling protected health information needs a risk analysis and Security Rule safeguards, the minimum-necessary standard where it applies, and, where a vendor handles the PHI for you as a business associate, a business associate agreement covering that use. Data de-identified under the Safe Harbor or Expert Determination method is not PHI, so those requirements do not apply to it.

    is ai allowed under hipaa

    AI and HIPAA: is AI allowed with patient data?

    Yes, HIPAA permits AI to be used with protected health information, on the same terms as any other system. That assumes the use is one HIPAA permits, such as treatment, payment or health care operations; state law and rules such as 42 CFR Part 2 can add requirements. What HIPAA does not do is approve products. Its rules bind covered entities (health plans, health care clearinghouses and health care providers that conduct certain transactions electronically, per 45 CFR 160.103) and their business associates, so the question is never whether a tool is approved but whether your use of it meets the rules. The three types of HIPAA covered entity are set out in plain terms in our covered entity guide.

    That is why the question 'which AI is approved for HIPAA' has no list for an answer. The same model can be used lawfully by one hospital and unlawfully by another. The difference is whether there is an agreement with the vendor and what it covers, how the tool is configured, which data reaches it, and the safeguards around it.

    A HIPAA and AI review comes down to four questions. Is the AI vendor a business associate, and if so is there an agreement whose permitted uses cover what the tool does? Has the system been covered by a risk analysis under the Security Rule? Where the minimum-necessary standard applies, is the tool limited to the PHI its purpose needs? And could the job be done with de-identified data instead, which takes that data outside the Privacy Rule's use and disclosure requirements? For what Microsoft's own pages say about Copilot and HIPAA, see Copilot and HIPAA: what Microsoft says.

    what hipaa requires of ai

    What AI HIPAA compliance requires, rule by rule

    Each row below is a requirement in the regulation text and what it means once an AI system is involved. Citations are to Title 45 of the Code of Federal Regulations.

    Is the Security Rule changing?

    HHS published a proposed rule to update the Security Rule in the Federal Register on 2025-01-06 (90 FR 898). As of October 2026 no final rule had been published in the Federal Register, so the technical safeguards at 164.312 summarized above are the rule as it reads today. Check the proposal's current status before you plan against any change it describes.

    RequirementWhere it isWhat it means for an AI system
    Business associate status45 CFR 160.103A person that, on your behalf, creates, receives, maintains or transmits PHI for a function or activity HIPAA regulates is a business associate, and so is a subcontractor that does the same for the business associate. The definition names services such as data analysis, processing or administration and billing, and consulting where the service involves disclosure of PHI. It has exclusions, such as a health care provider receiving PHI for treatment.
    Business associate agreement45 CFR 164.502(e), 164.504(e), 164.308(b)Before a business associate handles PHI for you, you need satisfactory assurances documented in a written contract or other arrangement. The contract must establish the permitted and required uses and disclosures and require the business associate to ensure its subcontractors agree to the same restrictions and conditions. No agreement is needed where no business associate is involved.
    Risk analysis45 CFR 164.308(a)(1)(ii)(A)An accurate and thorough assessment of risks to electronic PHI. A new AI system that holds or uses ePHI belongs in it.
    Access control45 CFR 164.312(a)Only people or software granted access rights may reach ePHI, each user needs a unique identifier, and encryption is an addressable specification.
    Audit controls45 CFR 164.312(b)Mechanisms that record and examine activity in systems that contain or use ePHI. The standard does not set a granularity; for AI we recommend a record per user, per patient record and per action. Those records can contain ePHI themselves.
    Transmission security45 CFR 164.312(e)Guard ePHI sent over a network, including calls to a model endpoint outside your environment.
    Minimum necessary45 CFR 164.502(b), 164.514(d)When using or disclosing PHI, or requesting it from another covered entity or business associate, make reasonable efforts to limit it to the minimum necessary to accomplish the intended purpose (164.502(b)). Exceptions are listed at 164.502(b)(2), including disclosures to or requests by a health care provider for treatment; 164.514(d) sets out how to meet the standard. How it applies to a given AI workflow is a question for your privacy officer or counsel.
    De-identification45 CFR 164.514(a)-(b), 164.502(d)Data de-identified by Safe Harbor or Expert Determination is not PHI, and the Privacy Rule's use and disclosure requirements do not apply to it.
    Breach notification45 CFR 164.402 (definitions); 164.404-164.410 (notification)An acquisition, access, use or disclosure of PHI the Privacy Rule does not permit is presumed to be a breach unless the covered entity or business associate demonstrates a low probability that the PHI has been compromised, based on a risk assessment of at least four factors. The definition has three exclusions, such as an unintentional acquisition, access or use by a workforce member made in good faith and within the scope of authority that does not lead to further use or disclosure the Privacy Rule does not permit. Notification duties attach to breaches of unsecured PHI.

    business associate agreements for ai

    Business associate agreements for AI vendors

    Where an AI vendor is a business associate, the agreement decides what it may do with your data, so start there. If the tool is general software rather than an AI product, read what HIPAA requires of software first. The regulation says what the contract must contain. It does not say which products a vendor's agreement covers; only the vendor's own terms do.

    If an AI vendor will handle PHI for you, then before the tool touches it, read the vendor's agreement and its product terms side by side and answer these questions in writing: which products, tiers and settings the agreement covers; whether prompts, files and outputs are retained, and for how long; whether the vendor may use them to train or improve its models; which subcontractors, including model providers behind the product, receive PHI and whether they are bound to the same terms; and how the vendor will report a use or disclosure the agreement does not permit.

    Vendor programs change, and a consumer or self-serve tier of a product can sit outside an agreement that covers the enterprise tier. We do not list which AI vendors offer agreements on which products here: check the vendor's current terms, dated, before you rely on them. For one product in detail, see our notes on ChatGPT Enterprise.

    de identified data and ai

    Using de-identified data with AI: Safe Harbor and Expert Determination

    If an AI use case can run on de-identified data, HIPAA's Privacy Rule use and disclosure requirements stop applying to that data. 45 CFR 164.514(b) gives two ways to get there.

    Safe Harbor

    Remove the eighteen identifier types listed at 164.514(b)(2)(i), from names, geographic units smaller than a state (with a narrow exception for the first three digits of a zip code) and all elements of dates other than the year, through device identifiers, URLs, IP addresses, biometric identifiers and full-face photographs, to any other unique identifying number or code. You must also have no actual knowledge that what remains could identify the person.

    Expert Determination

    A person with appropriate knowledge of and experience with generally accepted statistical and scientific principles and methods determines that the risk is very small that the information could be used, alone or combined with other reasonably available information, by an anticipated recipient to identify someone, and documents the methods and results of the analysis.

    Where de-identification goes wrong with AI

    Clinical free text is harder: names, dates and addresses sit inside notes, not in tidy columns, so a field-by-field scrub can miss them. A code that lets you re-identify records counts as PHI if it is disclosed (164.502(d)(2)(i)), and a vendor that does the de-identifying for you receives PHI to do it, so it is a business associate. Treat a de-identification step as a control to test, not a setting to switch on.

    ai vendor risk assessment gaps

    Where AI vendors fall short in a HIPAA security risk assessment

    When an AI vendor's tool comes up short in a HIPAA security risk assessment, look at four things: where the vendor is a business associate, whether the agreement's permitted uses cover what the AI tool now does; whether the vendor's terms let it retain prompts and outputs or train on them; whether activity is recorded per user and per patient record, so you can show who saw what; and whether the tool can reach more patient data than the same user could see in the EHR. Our page on why hospital AI vendors fail HIPAA risk assessments takes each one in turn.

    auditing ai agent decisions

    What to record when an AI agent acts on patient data

    The audit-controls standard at 45 CFR 164.312(b) asks for mechanisms that record and examine activity in information systems that contain or use ePHI. It does not set a level of detail. For an AI agent we recommend recording, for every AI-assisted action, which user triggered it, which patient record it touched, what went in, each tool call the agent made, what it produced and who reviewed the output before it reached a chart or a patient. The review record is the control that keeps a named person accountable for the decision. Those records can themselves contain PHI, so protect and retain them under the same safeguards. Our page on auditing an AI agent's decisions in a clinical setting sets out the design.

    hipaa ai audit

    HIPAA AI Audit Readiness Checklist

    A HIPAA AI audit readiness checklist has to cover the same ground as a standard HIPAA risk assessment, plus the failure points specific to how AI systems handle protected health information. The items below are the AI-specific checks we add to a standard HIPAA risk analysis.

    1. 1. Map every PHI data flow

      Document exactly where protected health information enters, moves through, and exits the AI system, including any third-party model API or subprocessor.

    2. 2. Confirm business associate agreement coverage

      Where the AI vendor is a business associate, verify a signed BAA's permitted uses cover what this AI system does with PHI, not just a general vendor relationship signed before the tool existed.

    3. 3. Run a system-specific risk analysis

      Do not rely on a reused organization-wide risk assessment. The AI system's data flow needs its own analysis under the Security Rule.

    4. 4. Scope access by role

      Confirm the AI tool restricts each user to the same patient data they would be authorized to see through the EHR directly.

    5. 5. Build per-user, per-patient audit logs

      Log which user, which patient record, what input, and what output for every AI-assisted action, as evidence for the 164.312(b) audit-controls standard.

    6. 6. Document the human review checkpoint

      Record who reviewed each AI output before it affected care, and what basis they used to accept or override it.

    7. 7. Confirm data retention and training-use limits

      Verify the vendor's terms do not let it retain or train on your PHI beyond what the BAA and Privacy Rule authorize.

    8. 8. Verify encryption in transit and at rest

      Confirm transmission security controls cover PHI moving to and from any AI model endpoint, not just internal systems.

    9. 9. Extend the breach response plan

      Update incident response procedures to cover AI-specific failure modes, including prompt leakage and over-broad logging.

    10. 10. Train staff on AI-specific PHI handling

      Workforce training should cover what can and cannot go into an AI tool.

    11. 11. Keep governance documentation current

      Maintain program-level documentation, aligned to a framework like the NIST AI RMF, that shows leadership oversight of the AI program as a whole.

    where hipaa stops

    Where HIPAA Stops and Other Frameworks Pick Up

    HIPAA governs how protected health information is handled. It does not govern whether an AI model's clinical output is accurate, unbiased, or well-validated, and that gap is where an AI system that meets every HIPAA safeguard can still make a bad clinical call.

    A system can satisfy every Security Rule technical safeguard, have a signed BAA, and pass a HIPAA audit, while still producing an unreliable clinical recommendation. HIPAA was written to protect data, not to validate model performance, so compliance with it is necessary and not remotely sufficient for clinical AI safety on its own.

    Where FDA rules apply, FDA oversight of AI that is itself a regulated device, and 21 CFR Part 11's electronic records and signatures requirements for FDA-regulated records, can layer on top of HIPAA, particularly in life sciences and laboratory settings where many teams follow GAMP 5 as their validation method.

    The NIST AI Risk Management Framework is voluntary guidance, and it is one framework that addresses questions HIPAA's text is silent on, like drift monitoring, model change control, or ongoing human oversight of an agentic system. Treat it as the operational layer that answers the how, where HIPAA only answers the what.

    how kriv ai works on hipaa and ai

    How Kriv AI helps with AI HIPAA compliance

    Kriv AI does not sign business associate agreements.

    A HIPAA and AI engagement starts with a list of the AI tools your organization uses, not with the model. We map each tool and mark which vendors are business associates and whether an agreement covers what each tool does. We run a gap check of each AI system against the Security Rule safeguards, whether it is a single model, a retrieval tool or a multi-step agent. We design the audit records and the human review step so the evidence exists before anyone asks for it. And we write the rules to the policy and frameworks you already answer to, such as HIPAA and the NIST AI RMF.

    We are a consultancy, not a law firm or a certifying body. Your privacy officer and counsel decide what HIPAA requires of your organization; we make sure the AI systems and the paperwork match what they decide.

    engagement

    How engagements are scoped and priced

    Every engagement is scoped and priced after a short discovery call.

    getting started

    Getting Started

    A useful first step in finding out where your AI systems stand against HIPAA is a scoped data-flow and risk conversation, not a generic AI policy template.

    That conversation maps which of your AI vendors are business associates, where the gaps sit against the Security Rule's technical safeguards, and what a defensible audit trail would need to look like for the specific systems you already have in production. See our healthcare and life sciences AI work and the AI readiness and governance assessment for how this fits into a broader program.

    Sources

    Cited sources

    Straight answers

    Frequently asked questions about AI HIPAA compliance: what HIPAA requires before AI touches PHI

    Is AI allowed under HIPAA?

    Yes. HIPAA has no rule against AI and no AI-specific rule. It applies the Privacy, Security and Breach Notification Rules to any system that handles protected health information. An AI tool can be used with PHI for a purpose HIPAA permits, such as treatment, payment or health care operations, when the system is covered by a risk analysis and safeguards, the minimum-necessary standard is met where it applies, and, where the vendor is a business associate, a business associate agreement covers the use. State law and rules such as 42 CFR Part 2 can add requirements.

    Is there an AI that can be used with PHI?

    HIPAA does not approve or list products, so no tool qualifies on its own. A tool can be used with PHI when your organization applies Security Rule safeguards around it and, where its vendor handles PHI for you as a business associate, the vendor's business associate agreement covers the specific product, tier and settings you use. Vendor programs change and may cover only some products or tiers, so check the vendor's current terms and date your check.

    Is there an AI model built for HIPAA?

    No model meets HIPAA by itself, because HIPAA regulates organizations and their handling of PHI, not model weights. The same model can be used lawfully in one deployment and unlawfully in another. What matters is the hosting arrangement, whether a business associate agreement is needed and what it permits, retention and training terms, access controls, audit records and whether the data reaching the model needs to be identifiable at all.

    What does HIPAA require before an AI tool handles PHI?

    A risk analysis that includes the system (45 CFR 164.308(a)(1)(ii)(A)). Technical safeguards: access control, audit controls, integrity, person or entity authentication and transmission security (164.312). The minimum-necessary standard where it applies (164.502(b)). And, where a vendor is a business associate, an agreement whose permitted uses cover what the tool does (164.504(e)). Data de-identified under 164.514 is not PHI, so the Privacy Rule's use and disclosure requirements do not apply to it.

    Do we need a business associate agreement with an AI vendor?

    If the vendor is a business associate, yes. A person that creates, receives, maintains or transmits PHI on your behalf for a function HIPAA regulates is a business associate under 45 CFR 160.103, whatever it calls itself, and the definition reaches services such as data analysis and billing, and consulting that involves PHI, as well as clinical tools. It has exclusions, such as a health care provider receiving PHI for treatment. Where an agreement is needed, you need it before PHI reaches the vendor. If the tool only ever receives properly de-identified data, it is not handling PHI.

    Can we use AI on de-identified health data?

    Yes. Data de-identified by Safe Harbor (removing eighteen identifier types, with no actual knowledge that the rest could identify someone) or by Expert Determination is not PHI, so the Privacy Rule's use and disclosure requirements do not apply to it. Test the de-identification on free text, keep any re-identification code inside your organization, and remember a vendor that de-identifies for you is a business associate.

    Does Kriv AI sign business associate agreements?

    No. Kriv AI does not sign business associate agreements. Our work is on the contracts' coverage, the controls and the documentation around your AI tools. Bring your use case to a discovery call and we will say plainly whether we are the right fit.

    Does HIPAA compliance mean an AI system is clinically safe?

    No. HIPAA governs how protected health information is handled, not whether a model's clinical output is accurate or well-validated. A system can meet every Security Rule control and still produce an unreliable recommendation, which is why frameworks like the NIST AI Risk Management Framework and, where FDA rules apply, FDA oversight of regulated devices and 21 CFR Part 11 for FDA-regulated electronic records sit alongside HIPAA rather than being replaced by it.

    Talk to the team that would do the work

    Bring your requirements to a working session with the person who'll actually deliver.

    Book a Discovery Call