Prompt injection is an attack in which crafted input causes a language model to follow instructions from an attacker instead of, or in addition to, the instructions intended by its developer or user. In AI safety and security, it is regarded as one of the most serious vulnerabilities of applications built on large language models, because the model cannot reliably distinguish instructions from data when both arrive as text in the same context.
The name deliberately echoes SQL injection, where data is executed as code. The analogy is imperfect: SQL injection can be eliminated with parameterized queries, while prompt injection has no complete technical fix today, because natural language has no strict boundary between command and content.
How prompt injection works
A language model application assembles its prompt from several sources: a system prompt written by the developer, the user’s message and often external content such as web pages, emails, documents or tool results. The model processes all of it as one sequence of tokens. If any part contains text phrased as an instruction, the model may treat it as one, regardless of where it came from.
An attack therefore needs only two conditions: attacker-controlled text reaches the context, and the model gives that text authority it should not have. The consequences grow with the capabilities of the application. A chatbot may merely produce wrong output, while an agent with tool access may send data, change files or make purchases on the attacker’s behalf.
The two main forms differ in who delivers the malicious text, as the following table shows.
| Form | Delivery path | Typical example |
|---|---|---|
| Direct injection | Attacker types into the application | Asking a support bot to reveal its system prompt |
| Indirect injection | Hidden in content the model reads | Instructions concealed in a web page or email |
| Stored injection | Planted in data retrieved later | Poisoned document in a retrieval index |
| Multimodal injection | Embedded in images or audio | Text rendered inside an uploaded image |
Prompt injection compared with related threats
Prompt injection is often confused with AI jailbreaks, but the two target different parties. A jailbreak tries to make a model break its own safety policy, usually for the benefit of the person typing. Prompt injection hijacks an application built on the model, typically against the interests of its user or operator.
| Threat | Goal of the attacker | Main victim |
|---|---|---|
| Prompt injection | Override the application’s instructions | User or operator |
| AI jailbreak | Bypass the model’s safety training | Model provider’s policy |
| Data poisoning | Corrupt behavior through training data | Model developer |
| System prompt extraction | Reveal hidden instructions | Application developer |
| Data exfiltration | Leak private information | User whose data is in context |
In practice these threats combine. An indirect injection may be used to exfiltrate data, for example by instructing the model to embed private details in a link or image URL that the interface then loads automatically.
Why agents raise the stakes
The risk of prompt injection scales with what the model is allowed to do. Security practitioners describe a dangerous combination of three capabilities: access to private data, exposure to untrusted content, and an ability to communicate externally. When a system has all three, a single injected instruction can move private data to an attacker.
Computer-use agents and coding agents are especially exposed, because they read arbitrary web pages, repositories and documents while holding powerful permissions. A comment in a code file or invisible text on a website can become an instruction the agent follows, which is why untrusted content must be treated strictly as data.
| Application type | Untrusted input | Potential impact |
|---|---|---|
| Plain chatbot | User messages only | Misleading or offensive output |
| Retrieval assistant | Retrieved documents | Manipulated answers |
| Email or browsing assistant | Messages and web pages | Data leakage, unwanted messages |
| Agent with write tools | Any content it reads | Unauthorized actions and transactions |
Defenses and mitigations
Because no single control solves the problem, defense relies on layered mitigations that reduce both the probability and the impact of a successful injection. The most effective measures limit what a compromised model can do rather than trying to detect every malicious phrase.
| Mitigation | How it helps | Limitation |
|---|---|---|
| Least privilege | Restricts tools and data per task | Reduces usefulness if too strict |
| Human confirmation | Approval before sensitive actions | Approval fatigue |
| Content separation | Marks untrusted text as data | Models may still obey it |
| Injection classifiers | Detect suspicious instructions | Bypassed by novel phrasing |
| Output filtering | Blocks links, markup and leaks | Narrow coverage |
| Dual-model designs | Privileged model never reads raw untrusted text | Complex to build |
Model developers also train models to prioritize instructions by source, giving system and developer messages more authority than tool output. This instruction hierarchy lowers attack success rates but does not eliminate them, so application builders should assume that some injections will succeed and design for containment. Regular AI red teaming with realistic injected content is the standard way to measure residual risk.
Common misconceptions
- A stronger system prompt is not a defense: telling the model to ignore injected instructions helps only marginally.
- Keyword filters are not sufficient: attacks can be paraphrased, translated, encoded or hidden in images.
- Hidden text is not harmless: white-on-white or zero-size text is invisible to people but fully visible to the model.
- Trusted sources can still carry injections: a legitimate site may host user comments or compromised content.
In plain words
Prompt injection is like slipping a forged note into a stack of papers that an assistant is told to read. The assistant cannot always tell the note apart from the genuine instructions from the boss, so it may obey the note instead.
AI systems read everything as text, including emails and web pages. If one of those contains hidden orders, the AI may follow them. The safest systems limit what the AI can do and ask a person before anything important happens.
A real-world example
A company gives employees an assistant that summarizes incoming email and can draft replies and search the internal file store. An outsider sends a message containing hidden white text that instructs the assistant to search for files about salaries and include their contents in a reply to the sender.
Because the security team had applied least privilege and human confirmation, the assistant could draft a reply but not send it, and outgoing drafts with attachments required explicit approval. The employee saw an unexpected draft addressed to an unknown sender and reported it. The injection technically succeeded, but its impact was contained, and the team added a classifier that flags emails with hidden text before the assistant processes them.