Supervised fine-tuning (SFT) is the training stage in which a pretrained model is further trained on labeled input and output pairs, so that it learns to produce the desired response for a given input. In artificial intelligence, and especially in the training of large language models, SFT is usually the first step of post-training: it turns a model that only predicts the next token of raw text into one that follows instructions, answers in a chosen format and adopts a consistent style.
The method is called supervised because every example carries an explicit target answer written or approved by people, or generated and filtered by another model. The model is not rewarded or ranked; it is simply taught to imitate demonstrations, which makes SFT predictable, cheap relative to pretraining and the foundation on which preference methods such as RLHF and DPO are built.
How supervised fine-tuning trains a model
Supervised fine-tuning works by minimizing the cross-entropy loss on target tokens while starting from the weights of an already pretrained model. Each training example contains a prompt, for example a user question, and a completion, for example the ideal answer. The two are concatenated into one sequence and passed through the network with teacher forcing, so the model sees the correct previous tokens at every position.
A key detail is loss masking: the loss is usually computed only on the completion tokens, not on the prompt. The model therefore learns to produce good answers rather than to reproduce questions. Gradients flow back through the network and an optimizer, typically the Adam optimizer with weight decay, updates the parameters with a learning rate far smaller than in pretraining.
The process runs for a small number of passes over the data, commonly one to five epochs. More passes on a small dataset quickly lead to memorization, because the model already has enormous capacity and the dataset is tiny compared with its pretraining corpus.
The following table lists the main hyperparameters of a typical SFT run for a language model and what each one controls.
| Parameter | Common range | What it affects |
|---|---|---|
| Learning rate | 1e-6 to 2e-5 (full); 1e-4 to 3e-4 (LoRA) | Speed of adaptation and risk of forgetting |
| Epochs | 1 to 5 | Fit to the data versus memorization |
| Batch size | 32 to 512 sequences | Gradient noise and training stability |
| Sequence length | 2,048 to 32,768 tokens | Longest example the model can learn from |
| Warmup | 1 to 5 percent of steps | Stability of the first updates |
| Loss masking | Completion tokens only | Whether prompts are learned or ignored |
The SFT dataset and its quality
The dataset is the decisive factor in supervised fine-tuning, because the model will imitate whatever the demonstrations show, including their mistakes. Experience across many projects shows that a few thousand carefully curated examples often outperform hundreds of thousands of noisy ones, a finding sometimes summarized as quality over quantity.
Data comes from several sources, each with its own strengths and weaknesses, as the table below summarizes.
| Source | Strength | Main weakness |
|---|---|---|
| Human-written demonstrations | High quality and control of style | Slow and expensive to produce |
| Converted public datasets | Large volume and task variety | Rigid templates and short answers |
| Synthetic data from a stronger model | Cheap and scalable | Inherits errors and limits of the source model |
| Rejection-sampled outputs | Verified correctness for math or code | Requires a reliable checker or reward model |
| Production logs with corrections | Matches real user requests | Privacy review and consent are required |
Good SFT datasets share a few properties: diverse tasks and phrasings, answers that are factually correct and consistently formatted, balanced lengths and the removal of near-duplicates. Data labeling guidelines, inter-annotator review and automatic filters are standard tools for keeping quality high. Methods such as Self-Instruct and rejection sampling fine-tuning are ways of producing SFT data at scale.
SFT compared with related training methods
Supervised fine-tuning differs from neighboring methods mainly in what signal the model learns from. SFT learns from correct examples, whereas preference methods learn from comparisons between better and worse answers, and pretraining learns from unlabeled text.
| Method | Training signal | Typical role |
|---|---|---|
| Pretraining | Next-token prediction on raw text | Builds general knowledge and language ability |
| Supervised fine-tuning | Target answers for given inputs | Teaches format, tasks and instruction following |
| Instruction tuning | SFT on instruction and response pairs | A common special case of SFT |
| RLHF | Reward model trained on human preferences | Refines helpfulness and harmlessness after SFT |
| DPO | Pairs of preferred and rejected answers | Simpler preference optimization without a reward model |
| Distillation | Outputs or probabilities of a teacher model | Transfers ability into a smaller student |
The boundaries overlap. Instruction tuning is SFT applied to instruction data, and distillation becomes SFT when the student learns from the teacher’s generated text rather than its probability distribution. SFT also differs from continued pretraining, which adds domain text without targets and changes knowledge more than behavior.
Full fine-tuning versus parameter-efficient SFT
Supervised fine-tuning can update all model weights or only a small added set of parameters. Full fine-tuning gives the most capacity to change behavior but needs memory for the weights, gradients and optimizer states, often three to four times the model size in mixed-precision training.
Parameter-efficient methods reduce this cost sharply. LoRA freezes the original weights and trains small low-rank matrices, while QLoRA additionally keeps the frozen base in 4-bit quantization, which allows a model with tens of billions of parameters to be fine-tuned on a single high-memory GPU.
| Situation | Recommended approach | Reason |
|---|---|---|
| Limited GPU memory | QLoRA | Lowest memory footprint with modest quality loss |
| Many task-specific variants | LoRA adapters | Small adapters can be swapped on one base model |
| Deep change of behavior or language | Full fine-tuning | All parameters can adapt |
| Small dataset under 1,000 examples | LoRA with few epochs | Limits overfitting and forgetting |
Limitations and common mistakes in SFT
Supervised fine-tuning mostly shapes behavior rather than adding knowledge. Facts that the pretrained model never saw are learned poorly from a few examples, and training on unfamiliar facts can even increase hallucination, because the model learns to answer confidently beyond what it knows.
- Catastrophic forgetting: a high learning rate or too many epochs erode general abilities learned in pretraining.
- Overfitting to style: the model copies surface patterns such as fixed openings or a single answer length.
- Exposure bias: with teacher forcing the model never sees its own errors during training, so mistakes can compound at inference.
- Imitation ceiling: SFT cannot exceed the quality of its demonstrations, which is one reason preference training follows it.
- Contaminated evaluation: benchmark questions that leak into SFT data inflate reported scores.
Practitioners counter these risks with held-out validation sets, early stopping, mixing some general data into the domain data and evaluating both the target task and broad capabilities after each run.
Understanding it simply
A pretrained language model has read an enormous amount of text, but it does not yet know how to act as a helpful assistant. Supervised fine-tuning is like giving a well-read new employee a binder of model answers: here is a question, here is exactly how we reply.
The model studies these examples and adjusts itself to answer in the same way. It does not learn the world from scratch; it learns how to use what it already knows in the format and tone people want. The better the examples, the better the result.
Applied in practice
A hospital network wants an assistant that turns clinicians’ dictated notes into structured discharge summaries. A general open-weight model with 8 billion parameters writes fluent text but ignores the required headings and sometimes invents medication details.
The team assembles 4,000 de-identified note and summary pairs, each reviewed by a clinical documentation specialist. They run SFT with LoRA at a learning rate of 2e-4 for three epochs, masking the loss on the notes, and keep 400 pairs as a validation set. Training takes about four hours on one GPU.
After fine-tuning, the share of summaries with every required section rises from roughly 60 to over 95 percent, and reviewers flag fewer unsupported medication entries. The team still keeps a human sign-off step, because SFT improved format and faithfulness but cannot guarantee factual accuracy.