Multi-task learning

Also known as: multitask learning

On this page 7

Multi-task learning (MTL), also written multitask learning, is a training paradigm in which a single model learns several related tasks simultaneously, sharing parameters and representations across them instead of training one isolated model per task. In artificial intelligence, it is one of the main ways to make training more data-efficient: signals from one task act as an inductive bias that helps the model generalize on the others.

The idea dates back to work on neural networks in the 1990s and now underlies much of modern training practice, from vision networks that detect, segment and estimate depth with one backbone to large language models whose instruction tuning mixes hundreds of task formats. Its central tension is simple to state: shared learning helps when tasks are related and hurts when they compete.

How multi-task learning shares knowledge

Multi-task learning works by forcing tasks to share part of the model, so that features useful for one task become available to the others. A typical network has a shared trunk (or encoder) that builds general representations, followed by task-specific heads that map those representations to each output, such as a class label, a bounding box or a numeric score.

Training minimizes a combined loss, usually a weighted sum of the per-task losses. Gradients from every task flow back into the shared layers, which therefore learn representations that serve all objectives at once. This acts as a form of implicit regularization: the shared layers cannot overfit to the noise of a single task without paying a penalty on the others.

The two classic architectures differ in how strictly parameters are shared. The table below summarizes them together with two widely used refinements.

DesignHow parameters are sharedTypical use
Hard parameter sharingOne common trunk, separate output headsVision backbones, most language models
Soft parameter sharingSeparate networks tied by a distance penalty or cross-linksLoosely related tasks with different inputs
Mixture of task expertsShared pool of experts with per-task gatingRecommender systems with many objectives
Adapters per taskFrozen shared model plus small task modulesAdding tasks cheaply after pretraining
Parameter-sharing designs in multi-task learning

Loss weighting and gradient conflicts

The hardest practical problem in multi-task learning is balancing the tasks, because losses differ in scale, difficulty and convergence speed. A regression loss measured in squared meters can dwarf a cross-entropy loss near 0.5, so a naive equal sum lets one task dominate the shared layers.

A second problem is gradient conflict: two tasks may push shared weights in opposite directions, which produces negative transfer, where joint training performs worse than separate models. Several families of methods address weighting and conflict, each with its own cost.

MethodCore mechanismMain trade-off
Static weightsFixed coefficients chosen by grid searchSimple but costly to tune
Uncertainty weightingLearns weights from homoscedastic task noiseCan down-weight hard but important tasks
GradNormEqualizes gradient magnitudes across tasksExtra backward computation
Gradient projection (PCGrad)Removes conflicting gradient componentsMemory grows with task count
Temperature samplingRebalances how often each task is sampledNeeds tuning of the temperature
Common task-balancing methods

Recent empirical studies report that carefully tuned static weights often match sophisticated balancing schemes, so a strong baseline should always be measured before adopting a complex method.

Multi-task learning differs from neighboring training concepts mainly in when tasks are learned and which ones matter. In transfer learning, knowledge flows in one direction from a source task to a target; in multi-task learning, all tasks are trained together and usually all of them are goals.

ApproachTask orderPrimary goal
Multi-task learningAll tasks at the same timeGood performance on every task
Transfer learningSource first, then targetPerformance on the target task
Continual learningTasks arrive one after anotherLearning new tasks without forgetting
Instruction tuningMany formatted tasks mixed togetherFollowing natural-language instructions
Auxiliary-task learningMain and helper tasks togetherPerformance on the main task only
How multi-task learning relates to neighboring concepts

Instruction tuning of language models is effectively multi-task learning at scale: a single objective, next-token prediction, is applied to a data mixture covering translation, summarization, question answering and coding. Because every task shares one output format, the weighting problem moves from the loss to the data mixture proportions.

When to use multi-task learning

Multi-task learning pays off when tasks share underlying structure and at least some of them have limited labeled data. It also reduces serving cost, since one model replaces several, which matters on mobile devices and in latency-sensitive systems.

SituationRecommended choiceReason
Related tasks, small datasetsJoint training with hard sharingShared features compensate for scarce labels
One main task, cheap side labelsAuxiliary tasks with low weightsSide tasks regularize the main one
Unrelated or competing tasksSeparate modelsAvoids negative transfer
Tight inference budgetShared backbone, light headsOne forward pass serves all outputs
Tasks added over timeAdapters or LoRA modules per taskNo need to retrain the shared model
Deciding whether to train tasks jointly

Task grouping is a useful middle path: measure pairwise transfer between tasks on a validation set, then train only mutually helpful tasks together and keep the rest apart.

Limitations and common mistakes

Multi-task learning is not a free improvement, and its failures are often silent until each task is evaluated separately. The most frequent problems are the following:

  • Negative transfer: an unrelated task degrades shared features and lowers accuracy on the others.
  • Reporting only the average score: a strong mean can hide a task that collapsed.
  • Unequal dataset sizes: the largest task dominates unless sampling is rebalanced.
  • Capacity limits: a small shared trunk cannot represent every task well, a problem sometimes called the curse of multilinguality in language models.
  • Harder debugging: a regression may come from any task, its data or its weight.

Robust practice therefore compares every task against a single-task baseline, tracks per-task validation curves and applies early stopping or checkpoint selection with all tasks in view.

Stripped to the essentials

Multi-task learning means teaching one model several jobs at once instead of building a separate model for each job. The jobs share most of the model, so what it learns for one job can help with the others.

Think of a student who studies physics and mathematics in the same term. Skills from one subject make the other easier. But if the student also takes an unrelated course that eats up all the study time, grades can drop. The same is true for models: related tasks help each other, unrelated ones can get in the way, so engineers must check each task separately.

Where it is applied

An engineering team building a driver-assistance camera needs lane detection, object detection and depth estimation from the same video stream. Running three separate networks would exceed the latency budget of about 30 milliseconds per frame on the vehicle computer.

The team trains one convolutional backbone with three task heads. Early experiments show depth estimation dominating training because its loss is roughly ten times larger, so they introduce uncertainty-based loss weighting and rebalance sampling for rare scenes such as night driving.

After tuning, the shared model matches the single-task detectors within one percentage point on each benchmark while lane detection improves slightly, because depth cues help separate road markings from shadows. The result is one network that serves all three outputs within the latency budget, at roughly a third of the original compute.