Activation steering

Also known as: steering vectors

On this page 7

Activation steering is a technique that changes the behavior of a neural network at inference time by adding a direction vector to its internal activations, rather than by changing its weights or its prompt. In AI safety, these steering vectors are used to make a language model more honest, less sycophantic or less willing to produce harmful content, and to test whether a concept is represented linearly inside the model.

The method sits between mechanistic interpretability and practical control. It builds on the observation that many high-level properties, such as sentiment, refusal or truthfulness, correspond to directions in the residual stream, so pushing activations along such a direction shifts the output in a predictable way.

How activation steering modifies the residual stream

Activation steering works by intercepting the hidden state at one or more transformer layers and adding a scaled vector before the computation continues. If h is the activation at layer L for a given token, the steered activation is h + α·v, where v is the steering vector and α is a steering coefficient that sets the strength and sign of the intervention.

The most common way to obtain v is contrastive activation addition. The practitioner prepares pairs of prompts that differ only in the target property, for example an honest and a deceptive answer to the same question, records the activations for both, and takes the mean difference between the two sets. Averaging over dozens or hundreds of pairs cancels unrelated variation and isolates the shared direction.

The steps below summarize a typical pipeline for building and applying a steering vector.

StepWhat happensMain design choice
Collect contrast pairsPrompts with and without the target behaviorNumber and diversity of pairs
Record activationsHidden states captured at chosen layersToken position, often the last token
Compute the directionMean difference or linear probe weightsNormalization of the vector
Select layer and scaleSweep over layers and coefficientsTrade-off between effect and fluency
Apply at inferenceVector added through a forward hookAll tokens or only generated tokens
Typical steps in a contrastive steering pipeline

Variants of steering vectors

Several families of steering methods exist, and they differ mainly in how the direction is found and how it is applied. Some are fully unsupervised, while others need labeled examples or a trained auxiliary model such as a sparse autoencoder.

VariantSource of the directionTypical use
Activation additionDifference between two single promptsQuick qualitative experiments
Contrastive activation additionMean difference over many prompt pairsHonesty, sycophancy, refusal control
Representation engineeringPrincipal component of contrast differencesReading and controlling concepts
Feature steeringDecoder direction of a sparse autoencoder featureTargeted, interpretable concepts
Directional ablationProjection that removes a directionTesting whether a concept is necessary
Main variants and how they derive the direction

Directional ablation deserves special attention because it is the reverse operation: instead of adding a vector, it projects out a direction from every activation. Research on refusal showed that removing a single direction can largely disable a model’s refusal behavior, which demonstrates both the power of the technique and its dual-use risk.

Activation steering compared with other control methods

Activation steering is one of several ways to shape model behavior, and its distinguishing feature is that it acts on internal representations at inference time while leaving weights untouched. This makes it cheap to try and easy to reverse, but less durable than training-based methods.

MethodWhat changesCostPersistence
PromptingInput text onlyVery lowPer request, easy to override
Activation steeringHidden states during the forward passLow, no gradient updatesPer deployment, removable
Fine-tuningModel weightsModerate to highPermanent until retrained
RLHF or preference trainingWeights via a reward signalHighPermanent, broad effect
Steering compared with prompting and fine-tuning

Compared with prompting, steering is harder for a user to override, because the intervention happens below the level of text that a prompt injection or jailbreak can reach. Compared with fine-tuning, it is more transparent, since the vector itself can be inspected, compared with other vectors and switched off instantly.

Choosing layers and steering strength

The effect of a steering vector depends strongly on where and how strongly it is applied. In practice, practitioners sweep layers and coefficients and measure both the target behavior and general capability on a held-out benchmark.

SituationRecommended choiceReason
Abstract behavior such as honestyMiddle layers of the networkHigh-level concepts are most linear there
Surface style or topicEarly to middle layersLexical features form earlier
Output becomes incoherentLower the coefficientActivations leave the normal distribution
Effect is too weakRaise the coefficient or add layersSignal is diluted by other features
Need minimal side effectsSteer only generated tokensPrompt encoding stays unchanged
Practical guidance for configuring a steering intervention

A useful rule is to express the coefficient relative to the typical activation norm at the chosen layer. Vectors that are small relative to that norm produce subtle shifts, while vectors several times larger tend to break fluency and produce repetitive or off-topic text.

Limitations and open questions

Activation steering is a promising research tool, but its reliability is still debated. Effects that look strong on a handful of prompts often turn out to be inconsistent across inputs, and some behaviors do not appear to have a clean linear direction at all.

  • Variable effectiveness: the same vector can work well on some prompts and have little or opposite effect on others.
  • Capability cost: strong steering can reduce accuracy on unrelated tasks, a local form of alignment tax.
  • Spurious directions: a contrast set may capture length, formatting or topic instead of the intended concept.
  • Dual use: the same method that adds a refusal direction can remove safety behavior from open-weight models.
  • Access requirement: steering needs white-box access to activations, so it is not available through most hosted APIs.

For these reasons, steering is best treated as a complement to training-based alignment, not a replacement. It is valuable for diagnosing what a model represents and for adjustable controls, while durable safety properties still rely on training, evaluation and guardrails.

Understanding it simply

Inside a language model, every word passes through many layers of numbers. Researchers found that some ideas, such as being polite or being honest, correspond to a particular direction in that space of numbers. Activation steering nudges the model along that direction while it is thinking, a bit like turning a dial on a mixing desk instead of rewriting the song.

Nothing in the model is permanently changed, and the dial can be turned back at any time. The catch is that turning it too far makes the output garbled, and the dial does not always do exactly what its label promises.

Applied in practice

A research team evaluating an open-weight chat model notices that it often agrees with users even when they state false facts, a pattern known as AI sycophancy. The team writes 300 contrast pairs in which the same question is answered once with agreement and once with a polite, accurate correction, then records activations at the middle layers of the model.

After sweeping layers and coefficients, they apply the negative of the sycophancy vector at one middle layer with a moderate coefficient. On a held-out set of 500 prompts, agreement with false statements drops noticeably, while scores on a general knowledge benchmark fall by only about one percentage point. At twice the coefficient, answers start to repeat phrases, so the team keeps the moderate setting and documents the vector, layer and scale alongside the model’s evaluation results.