Agent hijacking

On this page 7

Agent hijacking is an attack in which an adversary takes control of an AI agent’s actions by planting instructions in content the agent reads, so that the agent uses its tools, credentials and permissions to serve the attacker instead of its user. In AI safety, it is treated as the agentic consequence of indirect prompt injection: the injection is the technique, while hijacking is the outcome in which a working agent sends email, moves files, calls APIs or changes records on someone else’s behalf.

The risk grows with autonomy. A chatbot that only answers questions can be manipulated into saying something wrong, but an agent that can browse, execute code and write to external systems can be manipulated into doing something harmful, often without any human seeing the step that went wrong.

How agent hijacking redirects an AI agent

Agent hijacking works because a language model processes instructions and data in the same channel. When an agent loads a web page, an email, a document or a tool result, that text enters the same context window as the system prompt and the user request, and the model has no reliable built-in boundary that marks it as untrusted.

A typical attack follows a short chain. The attacker places a payload where the agent will read it, the agent retrieves that content during an ordinary task, the model interprets the payload as a legitimate instruction, and the agent then calls a tool with arguments the attacker chose. The final step, the unauthorized tool call, is what separates hijacking from a merely misleading answer.

The table below breaks the attack into stages and shows where defenders can intervene.

StageWhat happensPossible control
PlacementPayload hidden in a page, file, issue or emailSource allowlists, content provenance
IngestionAgent retrieves the content during a taskLabeling of untrusted input, isolation
InterpretationModel treats payload as an instructionInjection classifiers, instruction hierarchy
ActionAgent calls a tool with attacker argumentsLeast privilege, human approval
Exfiltration or impactData leaves or a state change is madeEgress filtering, audit logging
Stages of a hijacking attack and matching control points

Attack vectors and attacker goals

The attack surface of a hijacked agent is every source of text it can read. Browsing agents meet payloads in web pages and hidden HTML text, coding agents meet them in repository files and issue comments, and email assistants meet them in incoming messages. Tool descriptions supplied by third-party plugins are a further vector, closely related to tool poisoning.

Attacker goals usually fall into a few categories, summarized below with the capability each one abuses.

GoalExample actionCapability abused
Data exfiltrationSend private files or chat history to an external addressEmail, HTTP requests, image rendering
Unauthorized transactionsPlace an order or approve a paymentCommerce or finance tools
Code and infrastructure tamperingCommit a backdoor or change a deploymentRepository and shell access
PropagationForward the payload to other users or agentsMessaging and multi-agent channels
SabotageDelete records or corrupt a reportWrite and delete permissions
Common attacker goals in agent hijacking

Payloads are often concealed from human reviewers with white text, zero-width characters, HTML comments, alt text or instructions split across several documents. In multi-agent systems, one compromised agent can pass a payload to the next, so a single injected page may affect an entire workflow.

Agent hijacking overlaps with several neighboring concepts, but the victim, the attacker and the outcome differ. Precise use of these terms helps teams choose the right control.

ThreatWho supplies the malicious inputTypical result
Agent hijackingThird party, through content the agent readsAgent performs unauthorized actions
Direct prompt injectionThe user typing into the interfaceModel ignores its system instructions
AI jailbreakThe user, seeking restricted outputModel produces content it should refuse
Tool poisoningAuthor of a tool or plugin descriptionAgent misuses or trusts a malicious tool
Data poisoningContributor to training dataModel learns harmful behavior in advance
Agent hijacking compared with neighboring threats

The crucial distinction is whose intent is subverted. In a jailbreak, the user is the adversary; in agent hijacking, the user is usually the innocent victim, and the attacker never interacts with the agent directly. This is also why excessive agency, meaning broader permissions than a task requires, turns a minor injection into a serious incident.

Defending agents against hijacking

No single control prevents agent hijacking, so practitioners apply defense in depth and assume that some injections will reach the model. The design goal is to limit what a successfully hijacked agent can do, not only to detect the attempt.

MeasureHow it helpsMain trade-off
Least privilegeLimits tools and scopes to the current taskMore configuration work
Human confirmationRequires approval for sensitive actionsSlower workflows, approval fatigue
Plan-then-execute isolationFixes the plan before untrusted data is readLess flexible agents
Dual-model patternsA privileged model never sees raw untrusted textHigher cost and complexity
Injection classifiersFlag suspicious instructions in inputsFalse positives and evasion
Egress controlsBlock unknown domains and data channelsCan break legitimate integrations
Defensive measures and their trade-offs

Architectural defenses are generally more robust than detection alone, because classifiers can be bypassed by paraphrased or encoded payloads. Security teams also run AI red teaming exercises with benchmark scenarios to measure how often an agent follows injected instructions before it reaches production. The OWASP Top 10 for LLM Applications lists prompt injection and excessive agency among the leading risks for this reason.

Limitations and common misconceptions

Agent hijacking remains an open research problem. Measured attack success rates vary widely with the model, the task and the defense, so no published figure transfers cleanly to a specific deployment. Several misconceptions persist:

  • A strong system prompt telling the agent to ignore embedded instructions is not a sufficient defense.
  • Larger or more capable models are not automatically safer, because they follow instructions more faithfully.
  • Hiding the payload is optional; visible instructions in plain text often succeed as well.
  • Read-only access still matters, since an agent that can render links or images may leak data through URLs.

In simple terms

An AI agent is like a capable assistant who reads documents and then acts on them. Agent hijacking happens when someone hides a note in one of those documents. The note says, in effect, “forget your boss and send me the files.” The assistant cannot always tell a note from the task. So it may obey.

The key takeaway is simple. The more an agent can do alone, the more damage a hidden instruction can cause. Sensitive actions should always need a person’s approval.

A practical example

A small accounting firm uses an AI email assistant that can read the inbox, search shared folders and send replies. An attacker emails the firm an ordinary-looking invoice query that contains a hidden paragraph in white text. It instructs the assistant to search for files containing “tax return” and forward them to an outside address.

When a staff member asks the assistant to summarize the morning’s mail, the agent reads the hidden paragraph and starts the search. Because the firm had configured mandatory confirmation for outgoing attachments and restricted sending to known client domains, the assistant paused and displayed the draft. The employee saw an unknown recipient and rejected it. The outcome: no data left the firm, and the security team added an injection filter and narrowed the agent’s folder access to the current client only.