Skip to main content
    Kriv AI

    Healthcare AI

    Generative AI in healthcare: what it is and what the public sources say

    A plain answer from the World Health Organization, the US National Institute of Standards and Technology, the Food and Drug Administration and the Office of the National Coordinator for Health IT, in their own words.

    Generative AI in healthcare is generative AI used in health settings. NIST, the US National Institute of Standards and Technology, describes generative AI as AI models that "generate derived synthetic content". The World Health Organization (WHO) says large multi-modal models carry "documented risks of producing false, inaccurate, biased, or incomplete statements". In the United States, the Food and Drug Administration (FDA) says it "does not regulate AI as such; it regulates medical devices, including AI-enabled medical devices." Sources: WHO, NIST and FDA, each read 17 November 2026.

    plain answer

    Generative AI in healthcare is AI that works with several kinds of data

    WHO calls large multi-modal models (LMMs), AI that can take in text, images and video, "a type of fast growing generative artificial intelligence (AI) technology with applications across health care." It says LMMs "can accept one or more type of data inputs, such as text, videos, and images, and generate diverse outputs not limited to the type of data inputted."

    NIST published "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile" (NIST AI 600-1) on 26 July 2024. The profile gives a definition that it attributes to a US executive order: generative AI is "the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content. This can include images, videos, audio, text, and other digital content." NIST calls the profile "a cross-sectoral profile", so it is not written for healthcare alone.

    Generative AI is not the same thing as agentic AI that takes actions across systems. Our page on agentic AI in healthcare covers that case.

    plain answer

    WHO lists five uses of generative AI in health and the risks it documents

    WHO's guidance names five broad applications of LMMs for health: "Diagnosis and clinical care, such as responding to patients' written queries", "Patient-guided use, such as for investigating symptoms and treatment", "Clerical and administrative tasks, such as documenting and summarizing patient visits within electronic health records", "Medical and nursing education", and "Scientific research and drug development".

    It then says there are also "documented risks of producing false, inaccurate, biased, or incomplete statements, which could harm people using such information in making health decisions." A reader of generated health text needs to know this, because the statements can read as confident whether or not they are right. WHO adds that LMMs "may be trained on data that are of poor quality or biased, whether by race, ethnicity, ancestry, sex, gender identity, or age."

    WHO names two more risks. One is "'automation bias' by health care professionals and patients, whereby errors are overlooked that would otherwise have been identified or difficult choices are improperly delegated to a LMM." The other is that LMMs "are also vulnerable to cybersecurity risks that could endanger patient information".

    WHO says the need is for "engagement of various stakeholders: governments, technology companies, healthcare providers, patients, and civil society, in all stages of development and deployment of such technologies, including their oversight and regulation."

    plain answer

    NIST names confabulation as a risk of generative AI, with a healthcare example

    NIST uses the word confabulation for what many people call hallucination. Its definition is "The production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived."

    NIST gives a reason this happens: "Confabulations are a natural result of the way generative models are designed: they generate outputs that approximate the statistical distribution of their training data". It says such prediction "can produce factually accurate and consistent outputs" but "can also produce outputs that are factually inaccurate or internally inconsistent."

    NIST also gives a healthcare case: "a confabulated summary of patient information reports could cause doctors to make incorrect diagnoses and/or recommend the wrong treatments." It adds that outputs "may also include confabulated logic or citations that purport to justify or explain the system's answer, which may further mislead humans into inappropriately trusting the system's output."

    plain answer

    NIST also lists privacy, bias and over-reliance among its risks

    Among the risks NIST defines, Data Privacy covers "Impacts due to leakage and unauthorized use, disclosure, or de-anonymization of biometric, health, location, or other personally identifiable information or sensitive data." NIST says that "during adversarial attacks, LLMs have revealed sensitive information (from the public domain) that was included in their training data."

    Harmful Bias or Homogenization is defined as "Amplification and exacerbation of historical, societal, and systemic biases". On over-reliance, NIST says "Automation bias can exacerbate other risks of GAI, such as risks of confabulation or risks of bias or homogenization." This is the same automation bias that WHO names.

    These are NIST's general descriptions of risk. The profile is not a healthcare rule. NIST says the AI Risk Management Framework is "intended for voluntary use".

    plain answer

    In the US, FDA regulates medical devices, not AI as such

    FDA's page on artificial intelligence in software as a medical device says: "FDA does not regulate AI as such; it regulates medical devices, including AI-enabled medical devices." It also says "The term device includes software, except for certain software functions that are excluded from the device definition". Whether a given generative AI tool is a device depends on its intended use, which is a question for the vendor and its regulatory counsel.

    The same FDA page lists generative AI only in the titles of its resources: a discussion paper titled "Considerations for the Regulation of Generative AI-Enabled Medical Devices: Discussion Paper and Request for Feedback", a 2024 meeting on total product lifecycle considerations for generative AI-enabled devices, and a 2025 meeting on generative AI-enabled digital mental health medical devices.

    FDA's list of AI-enabled medical devices says the agency "will explore methods to identify and tag medical devices that incorporate foundation models". The part of that page we read does not say whether any listed device is LLM-based.

    plain answer

    US health IT rules set transparency requirements for AI and predictive algorithms

    The Office of the National Coordinator for Health IT (ASTP/ONC) page on its health data, technology and interoperability certification program says the HTI-1 rule sets "first of its kind transparency requirements for the artificial intelligence (AI) and other predictive algorithms", so that users can "assess such algorithms for fairness, appropriateness, validity, effectiveness".

    We did not read the rule text, so this page does not say which products it covers. The page also lists a fact sheet called "HTI-1 Decision Support Interventions and Predictive Models".

    limits

    What a reader can check about a generative AI tool, using only these sources

    These sources do not give a checklist, and this page does not claim one. They do point to questions a reader can ask a vendor or a project team. Each question below rests on one passage.

    Is the task narrow? WHO says developers should make sure LMMs "are designed to perform well-defined tasks with the necessary accuracy and reliability".

    What was tested before release, and what happens after? NIST says the focus of its Generative AI Public Working Group was limited to four considerations: "Governance, Content Provenance, Pre-deployment Testing, and Incident Disclosure". WHO asks governments for "mandatory post-release auditing and impact assessments, including for data protection and human rights".

    Is it a medical device? FDA says it regulates medical devices, and that the device definition includes software except for certain excluded functions. The vendor should be able to say how that applies to its tool.

    Are people checking the output? NIST says "Automation bias can exacerbate other risks of GAI"; WHO says automation bias leads to errors being overlooked. A named person who reviews generated text before it is used is a way to address that. For how an organization can test a model's outputs, see our page on LLM evaluation.

    Which rules apply to a given organization is a question for its privacy officer and counsel, not for this page.

    Straight answers

    Frequently asked questions about Generative AI in healthcare: what it is and what the public sources say

    What is generative AI in healthcare?

    WHO calls large multi-modal models "a type of fast growing generative artificial intelligence (AI) technology with applications across health care". NIST describes generative AI as AI models that "generate derived synthetic content". Sources: S1 and S7, read 17 November 2026.

    What are the risks of generative AI in healthcare?

    WHO names false, inaccurate, biased or incomplete statements and automation bias. NIST names confabulation, data privacy and harmful bias, among others. Sources: S1 and S7, read 17 November 2026.

    Does FDA regulate generative AI?

    FDA says it "does not regulate AI as such; it regulates medical devices, including AI-enabled medical devices." It lists a discussion paper on generative AI-enabled medical devices among its resources. Source: S3, read 17 November 2026.

    Is there a US rule on AI transparency in health IT?

    ASTP/ONC says its HTI-1 rule sets "first of its kind transparency requirements for the artificial intelligence (AI) and other predictive algorithms". We did not read the rule text. Source: S5, read 17 November 2026.

    Does this page say whether generative AI can be used with patient data?

    No. It does not. It reports what WHO, NIST, FDA and ASTP/ONC say. Your privacy officer and counsel decide what applies to your organization.

    Talk to the team that would do the work

    Bring your requirements to a working session with the person who'll actually deliver.

    Book a Discovery Call