Indirect prompt injection

On this page 7

Indirect prompt injection is an attack in which malicious instructions are planted in external content that a language model later processes, such as a web page, an email, a shared document or a tool result, so that the model follows the attacker rather than its user. In AI safety, it is regarded as one of the most serious security risks for LLM applications, because the attacker never needs access to the chat window.

The attack exploits a structural weakness: a large language model receives instructions and data in the same token stream and has no reliable built-in way to tell them apart. Once an assistant can browse, read files or call tools, every piece of text it ingests becomes a potential command channel.

How indirect prompt injection reaches the model

Indirect prompt injection works by poisoning the context window through a channel that the application trusts implicitly. The user asks a legitimate question, the application fetches outside content to answer it, and the fetched content carries text written to look like an instruction. The model then treats that text as part of its task.

A typical attack chain has four stages. The attacker plants a payload in content they control, the application retrieves the content during normal operation, the model interprets the payload as an instruction, and the model performs an action with the privileges of the victim, such as sending data or calling an API.

The following table lists the delivery channels most often abused in deployed systems.

ChannelHow the payload arrivesTypical target
Web pagesHidden text, HTML comments, white-on-white textBrowsing assistants and AI search
EmailsMessage body or quoted threadEmail summarizers and inbox agents
DocumentsText in PDFs, spreadsheets or slide notesCopilots in office suites
Retrieval corporaPoisoned chunks in a vector indexRAG chatbots
Tool outputsAPI responses, issue comments, code repositoriesCoding and workflow agents
Images and audioText rendered in pixels or spoken contentMultimodal models
Common delivery channels for injected instructions

Payloads are often invisible to the human user. Attackers use zero-width characters, tiny fonts, metadata fields or alt text, because the model reads the raw text while the person sees only the rendered page.

Direct versus indirect prompt injection

The difference lies in who supplies the malicious text. In direct prompt injection, the user types the adversarial instruction; in the indirect form, a third party hides it in data that the user did not write and usually never reads.

This distinction changes the threat model considerably, as the comparison below shows.

AttackSource of malicious textVictimMain goal
Direct prompt injectionThe userThe application ownerOverride system instructions
Indirect prompt injectionA third party via contentThe user and the operatorHijack actions or exfiltrate data
AI jailbreakThe userThe model provider’s policyObtain prohibited output
Tool poisoningA malicious tool descriptionAgents that load the toolSteer tool selection and calls
Data poisoningManipulated training dataEvery model userImplant persistent behavior
Comparison with related attacks in the AI safety topic

Indirect prompt injection is the more dangerous variant for agentic systems because the victim is an innocent user. The OWASP Top 10 for LLM Applications lists prompt injection, including the indirect form, as its first-ranked risk, and closely related agent hijacking describes the same pattern when autonomous agents are the target.

What attackers achieve with injected instructions

The impact of indirect prompt injection is bounded by what the model is allowed to do. A summarizer without tools can only distort its answer, while an agent with email, file and payment access can cause real-world damage on the user’s behalf.

OutcomeRequired capabilityExample effect
Data exfiltrationRendering links or images, outbound requestsPrivate chat history encoded into a URL
Unauthorized actionsTool or API accessForwarding emails, creating calendar invites
Output manipulationNone beyond text generationBiased product summary or false answer
Phishing deliveryText shown to the userFake security warning with a malicious link
PersistenceMemory or long-term storageInstruction saved for later sessions
PropagationAbility to send messagesSelf-replicating payload across inboxes
Attack outcomes and the capabilities they require

Researchers often describe the worst case as the lethal trifecta: an agent that combines access to private data, exposure to untrusted content and an outbound communication channel. When all three are present, a single poisoned document can be enough to leak sensitive information.

Defenses and how well they work

No single control eliminates indirect prompt injection, so practitioners rely on defense in depth. Mitigations fall into two families: those that try to make the model ignore embedded instructions, and those that limit the damage when the model is fooled anyway.

TechniqueMechanismMain limitation
Spotlighting and delimitersMarks untrusted text with tags or encodingModels can still be persuaded to obey
Instruction hierarchy trainingTeaches the model to rank system above dataReduces but does not remove success rates
Injection classifiersDetects suspicious text before it reaches the modelBypassed by novel or obfuscated payloads
Least privilegeGrants only the tools a task needsLimits usefulness of general agents
Human confirmationRequires approval for sensitive actionsApproval fatigue and rubber-stamping
Dual-model isolationA quarantined model reads untrusted data, a privileged one plansHigher cost and design complexity
Main mitigation techniques and their limits

Architectural patterns are the most robust option. Designs in which a privileged planner never sees raw untrusted text, and in which data flows are tracked so that tainted content cannot trigger sensitive tools, offer guarantees that do not depend on model behavior. Output controls such as blocking automatic image rendering and restricting outbound URLs close the most common exfiltration paths.

Common misconceptions and design mistakes

Many failed defenses share the same mistaken assumption that the model can be told to ignore attacks. A system prompt that says to disregard instructions in documents raises the bar slightly but is not a security boundary, because the same model that reads the rule also reads the attack.

  • Treating retrieved content as trusted because it comes from an internal index; anyone who can edit a wiki page or send an email can write into that index.
  • Relying on input filters alone, which attackers evade with paraphrase, translation, encoding or splitting the payload across chunks.
  • Granting broad OAuth scopes to an assistant, which turns every injection into a full account compromise.
  • Measuring robustness with a static test set; adaptive attackers routinely reach high success rates against defenses that score well on fixed benchmarks.
  • Assuming larger models are immune; more capable models follow instructions better, including malicious ones.

Red teams test agents with dedicated benchmarks and adaptive attack campaigns, and AI red teaming of this kind should be repeated whenever tools, data sources or model versions change.

Stripped to the essentials

An AI assistant that reads web pages or emails for you cannot easily tell the difference between the text it should summarize and the text it should obey. Indirect prompt injection abuses that confusion. Someone hides a sentence such as an order to send the user’s files elsewhere inside a page, and the assistant may carry it out.

It is like a personal secretary who opens your mail and follows any note that says it comes from you. The fix is not only a smarter secretary but fewer keys on the secretary’s ring and a rule that important actions need your signature.

Where it is applied

A mid-sized consulting firm deploys an email assistant that can read the inbox, search the document store and draft replies. During a security review, the internal red team sends a staff member an ordinary-looking invoice email with white text at the bottom instructing the assistant to search for files containing the word contract and include their contents in a link to an external address.

When the employee asks the assistant to summarize the day’s mail, the assistant follows the hidden text in roughly one in four trials. The team responds with three changes: the assistant may no longer render external links or images, sending any email requires explicit user confirmation, and content from outside the company is passed to a separate model that can only produce summaries. A repeated test campaign of 200 attempts produces no successful data exfiltration, although the assistant still occasionally repeats injected marketing claims in its summaries.