Instruction following

On this page 8

Instruction following is the capability of a model to carry out explicit directives written in natural language, including the task itself, its constraints, the required format and any priorities between competing requests. In language models it is the property that turns a raw text predictor into a usable assistant, and in artificial intelligence evaluation it is measured as a distinct skill, separate from knowledge or reasoning.

A model can know the answer and still fail the instruction: it may ignore a word limit, add unrequested commentary, switch languages or drop one of five requirements. Strong instruction following means the output satisfies every verifiable constraint while still serving the intent behind the request.

How models learn to follow instructions

Instruction following is learned after pretraining, not inherited from it. A base model trained only on next-token prediction continues text plausibly; asked “List three causes of inflation”, it may produce more questions instead of an answer, because that pattern also appears on the web.

Developers close this gap with post-training. In supervised fine-tuning, the model sees many pairs of instructions and high-quality responses, usually wrapped in a chat template that marks system, user and assistant turns. Preference optimization, through reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO), then rewards responses that people or reward models judge to be more compliant and helpful.

The following table summarizes what each training stage contributes to the final behavior.

StageTraining signalEffect on instruction following
PretrainingNext-token prediction on large corporaBroad knowledge, weak task compliance
Supervised fine-tuningCurated instruction and response pairsLearns turn structure and answer format
Preference optimizationRanked or paired responsesPrefers complete, constrained, helpful answers
Constraint-focused tuningSynthetic tasks with checkable rulesBetter adherence to counts, formats and languages
Training stages and their effect on instruction following

Types of instructions and constraints

Instructions differ in how easily compliance can be verified, and this difference shapes both evaluation and failure rates. Verifiable constraints such as “answer in under 100 words” or “return valid JSON” can be checked by a program, while semantic constraints such as “use a reassuring tone” require human or model judgment.

CategoryExample instructionHow compliance is checked
Task directiveSummarize the contract clausesHuman or model review
Length constraintUse at most three sentencesAutomatic count
Format constraintOutput a Markdown tableParser or pattern match
Language constraintReply only in SpanishLanguage detection
Content exclusionDo not mention competitor productsKeyword or semantic filter
Persona and toneRespond as a formal legal assistantRubric-based judgment
Multi-turn persistenceKeep using metric units from now onChecks across later turns
Common instruction categories

Real prompts usually combine several categories. Failure probability grows with the number of simultaneous constraints, so a request with six independent rules is far harder than six requests with one rule each.

Instruction hierarchy and conflicting directives

Modern assistants follow an instruction hierarchy that ranks sources of directives by trust. Developer-level text in the system prompt normally outranks user messages, and user messages outrank text found inside documents, web pages or tool outputs that the model merely reads.

This ordering is the main defense against prompt injection, where content embedded in retrieved data tries to issue new commands. A model with good instruction following obeys the right principal, not the most recent or most forceful sentence in its context window.

SourceTypical priorityReason
Platform and safety policyHighestSet by the model provider and not negotiable
System promptHighDefines the product, role and boundaries
User messageMediumExpresses the current task within those boundaries
Retrieved or tool contentLowestTreated as data, not as commands
Typical priority of instruction sources

When two instructions at the same level conflict, a well-behaved model resolves toward the later or more specific one, or asks for clarification when the conflict changes the outcome materially.

How instruction following is measured

Instruction following is measured with benchmarks that isolate compliance from correctness. IFEval uses prompts with programmatically checkable rules and reports strict and loose accuracy at the prompt and instruction level. Other suites test multi-turn persistence, complex multi-constraint prompts and system prompt adherence.

ApproachStrengthLimitation
Rule-based checksObjective and reproducibleCovers only mechanical constraints
Model-as-judge scoringHandles tone and intentJudge bias and self-preference
Human preference ratingCaptures real usefulnessCostly, slow and inconsistent
Production telemetryReflects actual usageNoisy and hard to attribute
Evaluation approaches compared

Scores are sensitive to prompt phrasing. A model that reaches high accuracy on a public benchmark can still drop several points when the same constraints are reworded, which is why teams build private evaluation sets that mirror their own prompts.

Instruction following overlaps with several neighboring ideas but is not identical to them. Steerability describes how far behavior can be shifted through prompts, especially style and persona, whereas instruction following concerns whether a given directive is executed faithfully.

  • Prompt engineering is the human practice of writing instructions; instruction following is the model property that makes those instructions effective.
  • In-context learning infers a task from examples, while instruction following executes a task that is stated explicitly.
  • Constrained decoding enforces format at the token level from outside the model, which guarantees syntax but not the meaning of the content.
  • Hallucination is a factual failure; a response can follow every instruction and still contain invented facts.

Common failure modes and how to reduce them

Most instruction failures fall into a small set of recurring patterns, and each has a practical mitigation on the prompt or system side.

FailureLikely causeMitigation
Dropped constraintToo many rules in one prose paragraphList requirements explicitly and number them
Instruction driftRule stated early in a long conversationRestate key rules or move them to the system prompt
Over-literal complianceRule followed against obvious intentExplain the purpose behind each rule
Format breakageFree-form generation of structured outputUse structured output or constrained decoding
Injected command obeyedWeak separation of data and instructionsDelimit untrusted content and label it as data
Counting errorsTokenization hides word and character unitsValidate length in code and retry
Failure patterns, causes and fixes

A key trade-off is between strict compliance and helpfulness. A model tuned to obey every literal rule can become brittle, while one tuned for helpfulness may quietly override constraints it considers unimportant. Production systems should verify outputs programmatically rather than assume compliance.

In plain words

Instruction following is how well an AI system does exactly what it was asked. If you ask for three bullet points in French, a good model gives you three bullet points in French, not five points in English with a friendly introduction.

Think of a capable new colleague. Broad knowledge is not enough; they also need to respect the brief, the deadline and the format. Knowledge and obedience to the brief are separate skills, and AI developers train and test them separately.

A real-world example

An insurance company builds a claims assistant that reads incident reports and must return a JSON object with exactly seven fields, a severity score from 1 to 5, no personal names and a summary under 60 words. In early testing, about one response in twelve breaks a rule: a missing field, a name copied from the report or a summary of 80 words.

The team moves the rules into a numbered system prompt, marks report text as untrusted data, enables structured output for the JSON schema and adds a validator that rejects summaries over the limit and triggers a retry. After these changes, format errors effectively disappear and the remaining issues are rare name leaks, which are caught by a separate redaction step before anything reaches the claims database.