Activation steering is a technique that changes the behavior of a neural network at inference time by adding a direction vector to its internal activations, rather than by changing its weights or its prompt. In AI safety, these steering vectors are used to make a language model more honest, less sycophantic or less willing to produce harmful content, and to test whether a concept is represented linearly inside the model.
The method sits between mechanistic interpretability and practical control. It builds on the observation that many high-level properties, such as sentiment, refusal or truthfulness, correspond to directions in the residual stream, so pushing activations along such a direction shifts the output in a predictable way.
How activation steering modifies the residual stream
Activation steering works by intercepting the hidden state at one or more transformer layers and adding a scaled vector before the computation continues. If h is the activation at layer L for a given token, the steered activation is h + α·v, where v is the steering vector and α is a steering coefficient that sets the strength and sign of the intervention.
The most common way to obtain v is contrastive activation addition. The practitioner prepares pairs of prompts that differ only in the target property, for example an honest and a deceptive answer to the same question, records the activations for both, and takes the mean difference between the two sets. Averaging over dozens or hundreds of pairs cancels unrelated variation and isolates the shared direction.
The steps below summarize a typical pipeline for building and applying a steering vector.
| Step | What happens | Main design choice |
|---|---|---|
| Collect contrast pairs | Prompts with and without the target behavior | Number and diversity of pairs |
| Record activations | Hidden states captured at chosen layers | Token position, often the last token |
| Compute the direction | Mean difference or linear probe weights | Normalization of the vector |
| Select layer and scale | Sweep over layers and coefficients | Trade-off between effect and fluency |
| Apply at inference | Vector added through a forward hook | All tokens or only generated tokens |
Variants of steering vectors
Several families of steering methods exist, and they differ mainly in how the direction is found and how it is applied. Some are fully unsupervised, while others need labeled examples or a trained auxiliary model such as a sparse autoencoder.
| Variant | Source of the direction | Typical use |
|---|---|---|
| Activation addition | Difference between two single prompts | Quick qualitative experiments |
| Contrastive activation addition | Mean difference over many prompt pairs | Honesty, sycophancy, refusal control |
| Representation engineering | Principal component of contrast differences | Reading and controlling concepts |
| Feature steering | Decoder direction of a sparse autoencoder feature | Targeted, interpretable concepts |
| Directional ablation | Projection that removes a direction | Testing whether a concept is necessary |
Directional ablation deserves special attention because it is the reverse operation: instead of adding a vector, it projects out a direction from every activation. Research on refusal showed that removing a single direction can largely disable a model’s refusal behavior, which demonstrates both the power of the technique and its dual-use risk.
Activation steering compared with other control methods
Activation steering is one of several ways to shape model behavior, and its distinguishing feature is that it acts on internal representations at inference time while leaving weights untouched. This makes it cheap to try and easy to reverse, but less durable than training-based methods.
| Method | What changes | Cost | Persistence |
|---|---|---|---|
| Prompting | Input text only | Very low | Per request, easy to override |
| Activation steering | Hidden states during the forward pass | Low, no gradient updates | Per deployment, removable |
| Fine-tuning | Model weights | Moderate to high | Permanent until retrained |
| RLHF or preference training | Weights via a reward signal | High | Permanent, broad effect |
Compared with prompting, steering is harder for a user to override, because the intervention happens below the level of text that a prompt injection or jailbreak can reach. Compared with fine-tuning, it is more transparent, since the vector itself can be inspected, compared with other vectors and switched off instantly.
Choosing layers and steering strength
The effect of a steering vector depends strongly on where and how strongly it is applied. In practice, practitioners sweep layers and coefficients and measure both the target behavior and general capability on a held-out benchmark.
| Situation | Recommended choice | Reason |
|---|---|---|
| Abstract behavior such as honesty | Middle layers of the network | High-level concepts are most linear there |
| Surface style or topic | Early to middle layers | Lexical features form earlier |
| Output becomes incoherent | Lower the coefficient | Activations leave the normal distribution |
| Effect is too weak | Raise the coefficient or add layers | Signal is diluted by other features |
| Need minimal side effects | Steer only generated tokens | Prompt encoding stays unchanged |
A useful rule is to express the coefficient relative to the typical activation norm at the chosen layer. Vectors that are small relative to that norm produce subtle shifts, while vectors several times larger tend to break fluency and produce repetitive or off-topic text.
Limitations and open questions
Activation steering is a promising research tool, but its reliability is still debated. Effects that look strong on a handful of prompts often turn out to be inconsistent across inputs, and some behaviors do not appear to have a clean linear direction at all.
- Variable effectiveness: the same vector can work well on some prompts and have little or opposite effect on others.
- Capability cost: strong steering can reduce accuracy on unrelated tasks, a local form of alignment tax.
- Spurious directions: a contrast set may capture length, formatting or topic instead of the intended concept.
- Dual use: the same method that adds a refusal direction can remove safety behavior from open-weight models.
- Access requirement: steering needs white-box access to activations, so it is not available through most hosted APIs.
For these reasons, steering is best treated as a complement to training-based alignment, not a replacement. It is valuable for diagnosing what a model represents and for adjustable controls, while durable safety properties still rely on training, evaluation and guardrails.
Understanding it simply
Inside a language model, every word passes through many layers of numbers. Researchers found that some ideas, such as being polite or being honest, correspond to a particular direction in that space of numbers. Activation steering nudges the model along that direction while it is thinking, a bit like turning a dial on a mixing desk instead of rewriting the song.
Nothing in the model is permanently changed, and the dial can be turned back at any time. The catch is that turning it too far makes the output garbled, and the dial does not always do exactly what its label promises.
Applied in practice
A research team evaluating an open-weight chat model notices that it often agrees with users even when they state false facts, a pattern known as AI sycophancy. The team writes 300 contrast pairs in which the same question is answered once with agreement and once with a polite, accurate correction, then records activations at the middle layers of the model.
After sweeping layers and coefficients, they apply the negative of the sycophancy vector at one middle layer with a moderate coefficient. On a held-out set of 500 prompts, agreement with false statements drops noticeably, while scores on a general knowledge benchmark fall by only about one percentage point. At twice the coefficient, answers start to repeat phrases, so the team keeps the moderate setting and documents the vector, layer and scale alongside the model’s evaluation results.