AI Agent Security Risk in the Enterprise

The OWASP Gen AI Security Project defines the agency problem plainly. An LLM-based system is often given "the ability to call functions or interface with other systems via extensions," and the decision about which extension to invoke may be delegated to the model. OWASP calls the resulting weakness Excessive Agency, the vulnerability that enables damaging actions "in response to unexpected, ambiguous or manipulated outputs from an LLM, regardless of what is causing the LLM to malfunction."

OWASP states that "the root cause of Excessive Agency is typically one or more of: excessive functionality; excessive permissions; excessive autonomy." Its examples are concrete: an agent given a document extension that can also modify and delete documents, a read-only extension that connects with an identity holding update and delete rights, and a deletion tool that runs without any confirmation. Each one is a decision someone made when wiring the agent up, which is why it can be reviewed and governed.

NIST's Center for AI Standards and Innovation describes agent hijacking as "a type of indirect prompt injection in which an attacker inserts malicious instructions into data that may be ingested by an AI agent, causing it to take unintended, harmful actions." The underlying problem, CAISI explains, arises when a system lacks "a clear separation between trusted internal instructions and untrusted external data." An email, a file, or a web page can carry instructions the agent treats as part of its task.

OWASP is candid that this cannot be fully prevented today. Its prompt injection entry says that given the stochastic nature of models, "it is unclear if there are fool-proof methods of prevention for prompt injection," and that retrieval and fine-tuning do not fully mitigate it. Enterprise security for agents therefore has to assume some injected instructions will get through and limit what they can do.

OWASP recommends limiting the extensions an agent can call to the minimum necessary, avoiding open-ended tools such as running a shell command or fetching any URL, and limiting each extension's permissions on other systems. Its example: an agent that only makes purchase recommendations may need read access to one products table and nothing else, enforced by database permissions on the identity the extension uses.

OWASP advises executing actions in the specific user's authorization context rather than a generic privileged account, and implementing "authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not." The model proposes, the downstream system enforces.

OWASP recommends using "human-in-the-loop control to require a human to approve high-impact actions before they are taken." Governance decides which actions count as high impact, such as payments, record changes, external messages, and anything touching regulated data, and builds the approval into the workflow with a log.

OWASP's prompt injection guidance includes segregating and clearly identifying external content and running regular adversarial testing, "treating the model as an untrusted user" to test trust boundaries and access controls. CAISI's hijacking work adds that evaluations need to be adaptive, because red teaming can reveal new weaknesses even after known attacks are addressed, and that task-specific results can tell you more than an aggregate score.