AI Security
AI data security: how data leaks from AI systems and what controls help
An AI system can pass sensitive data to the wrong place in four ways. Knowing the four paths is the first step to closing them.
AI data security is the work of keeping sensitive data from reaching people or systems that should not have it when AI is in the loop. A data leak happens through four paths: what people put into prompts, what the model writes back, the connectors that feed it data, and the logs it leaves behind. OWASP lists sensitive information disclosure as a risk for LLM applications, including leaked personal data and confidential business information. The controls start with deciding what data may enter the system, limiting access to the minimum needed, and keeping clear rules for how long data is kept.
plain answer
What is AI data security?
NIST's AI Risk Management Framework describes an AI system as secure when it can maintain confidentiality, integrity and availability through protection mechanisms that prevent unauthorized access and use. NIST lists among common security concerns "the exfiltration of models, training data, or other intellectual property through AI system endpoints."
A data leak is the confidentiality half of that. Information reaches someone it should not, often without any attacker involved, because the system was set up to move data further than anyone intended.
framework
Four paths an AI data leak can take
These four paths are a way of organizing the review. They are our grouping, not a list taken from a standard. Each one can be checked on its own.
Prompts
People paste records, contracts or code into an AI tool. Once the data is in a prompt, where it goes depends on the tool's settings and the vendor's terms. OWASP's guidance for this path includes advising users on avoiding sensitive input and keeping clear data retention, usage and deletion policies.
Outputs
OWASP says LLMs embedded in applications risk exposing "sensitive data, proprietary algorithms, or confidential details through their output." Its examples include personal information disclosed during interactions and generated responses that include confidential business information.
Connectors
When an AI tool is connected to files, mail, databases or other systems, it can read whatever the connection allows. OWASP's mitigation list includes limiting access to sensitive data based on the principle of least privilege and limiting the model's access to external data sources.
Logs
Prompts, outputs and error messages are often stored for debugging or monitoring. A log that holds sensitive text is another copy of it. OWASP's guidance on secure system configuration includes avoiding data leaks through error messages or configuration details. Treating logs as a leak path is our addition, not OWASP's.
how to
Controls to put in place
OWASP's Top 10 for LLM Applications entry on sensitive information disclosure lists these among its prevention and mitigation strategies. Which ones apply depends on the system.
Sanitize what goes in
Use data sanitization so user data does not enter the training model, and apply input validation to detect and filter sensitive inputs.
Limit access
Apply the principle of least privilege to the data a system and its users can reach.
Set retention and use rules
Keep clear policies on how data is retained, used and deleted, and let users opt out of their data being used for training where that is offered.
Lock down configuration
Limit users' ability to override or read a system's initial settings, and keep errors from exposing configuration details.
Redact before processing
OWASP also lists tokenization and pattern-based redaction of sensitive content before the data is processed.
ai gap
Security and privacy trade off against each other
NIST notes that privacy-related risks may influence security, bias and transparency, and come with tradeoffs with these other characteristics. Our own example: a control that logs everything helps an investigation and also creates a store of sensitive text. Decide these tradeoffs on purpose and write down who accepted them.
differentiation
How this differs from our related pages
This page is the general guide to data leaving through AI systems. Attacks that hijack a model through its inputs are covered on our prompt injection page, tool and connector security on our MCP security page, and model-level security on our LLM security page. Our AI data governance page covers who owns and approves data use.
kriv role
What Kriv AI does
We help regulated organizations map where data can leave an AI system, set the controls and write down the decisions. Every engagement is scoped and priced after a short discovery call.
Related resources
Continue exploring
Straight answers
Frequently asked questions about AI data security: how data leaks from AI systems and what controls help
What is AI data security?
Keeping sensitive data from reaching people or systems that should not have it when AI is involved. NIST describes a secure AI system as one that maintains confidentiality, integrity and availability.
How does data leak from an AI system?
Through what goes into prompts, what the model outputs, the connectors that feed it data and the logs it keeps. OWASP lists leaked personal data and confidential business information among the examples.
What controls reduce AI data leaks?
OWASP lists data sanitization, least-privilege access, clear retention and deletion policies, locked-down configuration, and redaction before processing.
Can an AI data leak happen without an attacker?
In our view, yes. A person pasting sensitive text into a prompt, or a connector with wider access than needed, can expose data with no attacker involved.
Talk to the team that would do the work
Bring your requirements to a working session with the person who'll actually deliver.
Book a Discovery Call