Teacher forcing

On this page 7

Teacher forcing is a training technique for autoregressive sequence models in which the model receives the ground-truth previous token as input at every step, rather than the token it predicted itself. In artificial intelligence training, it is the standard way to train recurrent networks, sequence-to-sequence models and decoder-only transformers on next-token prediction, because it turns one long dependent generation into many independent, parallelizable predictions.

The method makes training fast and stable, but it creates a gap between how the model is trained and how it is used. At inference time there is no teacher: the model must condition on its own outputs, and errors can accumulate. This mismatch, called exposure bias, is the central trade-off that every practitioner working with teacher forcing must understand.

How teacher forcing works during training

Teacher forcing works by shifting the target sequence one position to the right and using it as the decoder input. For a target sequence of tokens y1 to yT, the model receives a start token plus y1 to yT-1 and must predict y1 to yT. The loss at each position is computed independently, usually with cross-entropy loss between the predicted distribution and the true next token.

Because every input token is known in advance, the model never has to wait for its own prediction. In a transformer, a causal attention mask prevents each position from seeing future tokens, so all positions are processed in a single forward pass. In a recurrent network the steps are still sequential, but the hidden state at each step is driven by correct inputs, which keeps gradients well behaved early in training.

The table below traces one training example through the process for a short target sentence.

PositionDecoder inputTarget to predictSource of input
1start tokenTheFixed start symbol
2ThecatGround truth
3catsatGround truth
4satdownGround truth
5downend tokenGround truth
Inputs and targets for one training sequence

The term comes from early work on recurrent networks, where a teacher signal replaced the network’s output in its own feedback loop. The name stuck when sequence-to-sequence translation models adopted the same idea, and it now describes the default maximum likelihood training of nearly every language model.

Why teacher forcing dominates language model training

Teacher forcing dominates because it is computationally efficient and mathematically equivalent to maximizing the likelihood of the training data. The probability of a sequence factorizes into a product of conditional probabilities, and teacher forcing evaluates each factor with the true context, which is exactly what maximum likelihood estimation requires.

These properties explain why pretraining, supervised fine-tuning and instruction tuning all rely on it. The following table summarizes the practical advantages and what each one enables.

BenefitMechanismPractical effect
Parallel computationAll target positions known in advanceFull sequence processed in one pass
Stable gradientsInputs are always correctFaster convergence early in training
Simple objectivePer-token cross-entropyEasy to implement and to scale
Exact likelihoodChain rule factorizationPerplexity is directly measurable
Dense supervisionLoss at every positionMany learning signals per example
Practical benefits of teacher forcing

Without teacher forcing, a transformer trained on sequences of several thousand tokens would need thousands of sequential generation steps per example, which would make large-scale pretraining impractical on current hardware.

Exposure bias and the train-inference mismatch

Exposure bias is the discrepancy between training, where the model only sees correct prefixes, and inference, where it sees its own possibly flawed prefixes. A model trained purely with teacher forcing has never practiced recovering from its own mistakes, so a single early error can push it into a region of input space it rarely encountered.

The practical severity of exposure bias is debated among researchers. Small recurrent models on short tasks showed clear degradation, while large transformer language models appear far more robust, partly because massive and diverse training data covers many imperfect contexts. Repetition loops and drifting long outputs are often attributed to it, but decoding strategy and model capacity also contribute, so attributing a failure to exposure bias alone requires care.

The table below contrasts the two regimes that create the mismatch.

AspectTraining with teacher forcingFree-running inference
Input at each stepTrue previous tokenModel’s own previous token
Error propagationNone, errors do not carry overErrors can compound
ParallelismFully parallel in transformersStrictly sequential
Distribution of prefixesData distributionModel distribution
Typical failureNot visible in training lossDrift, repetition, incoherence
Training versus inference conditions

Alternatives and mitigations

Several methods reduce the gap between teacher-forced training and free-running generation, each with a cost in complexity or compute. None has replaced teacher forcing as the default; they are usually applied as an additional stage or a partial modification.

MethodCore ideaMain drawback
Scheduled samplingGradually replace true tokens with model samplesBreaks parallelism, biased objective
Professor forcingAdversarially align free and forced hidden statesUnstable, rarely used at scale
Sequence-level trainingOptimize a reward over full generated outputsHigh variance, expensive sampling
RLHF and RLAIFTrain on the model’s own sampled responsesRequires a reward model
Rejection sampling fine-tuningFine-tune on filtered self-generated outputsDepends on a reliable filter
Methods that address exposure bias

In modern large language models, post-training stages such as RLHF and DPO partly serve this role: the model is optimized on text it generated itself, which exposes it to its own distribution. Scheduled sampling remains common in smaller sequence models, for example in speech recognition and time-series forecasting, where sequential decoding is cheap enough.

Practical guidance and common mistakes

In practice, teacher forcing should be the default for pretraining and supervised fine-tuning, with evaluation always performed in free-running mode. A low teacher-forced loss does not guarantee good generation, so sampled outputs must be inspected alongside validation perplexity.

  • Off-by-one shifting errors: misaligning inputs and labels lets the model see the token it must predict, producing an implausibly low loss.
  • Missing causal mask: in a transformer, omitting the mask leaks future tokens and yields the same misleading result.
  • Loss on prompt tokens: in instruction tuning, masking the prompt so that the loss covers only the response is usually preferred.
  • Judging by loss alone: teacher-forced metrics overstate quality for long or open-ended generation.
  • Aggressive scheduled sampling: replacing too many true tokens early can stall convergence.

Teacher forcing interacts with neighboring choices such as label smoothing, which softens the one-hot targets, and curriculum learning, which can order examples so that the model handles short sequences before long ones.

In simple terms

Imagine a student learning to recite a long poem. During practice, a teacher whispers the correct previous line each time, so the student only has to produce the next line. Practice goes quickly and smoothly. On stage, however, nobody whispers, and one forgotten line can throw off everything that follows.

Teacher forcing works the same way. During training, an AI model always sees the correct earlier words and only guesses the next one. This makes training fast and reliable, but the model gets less practice in correcting itself when it later writes on its own.

A practical example

A machine learning team trains a transformer to translate customer support tickets from German into English. Each training pair contains a German ticket and a reference English translation of about 120 tokens. With teacher forcing, the decoder receives the reference translation shifted by one position, and all 120 predictions are computed in one forward pass per batch, so an epoch over two million pairs finishes in hours rather than days.

The validation loss looks excellent, yet human reviewers notice that long tickets sometimes end with repeated phrases. The team keeps teacher forcing for the main training run, adds a short fine-tuning stage on the model’s own sampled translations filtered by a quality score, and evaluates with beam search. Repetition on long tickets drops noticeably, while training cost rises only modestly because the expensive stage remains fully parallel.