Multi-label classification is the supervised learning task in which each example may belong to zero, one or several classes at once, so the model predicts a set of labels rather than a single category. In machine learning and applied artificial intelligence, it describes problems such as tagging a news article as both politics and economy, or marking a chest X-ray with several co-occurring findings.
The defining property is that labels are not mutually exclusive. This single difference changes the output layer, the loss function, the decision thresholds and the evaluation metrics, and it makes dependencies between labels a central modeling concern.
How multi-label classification produces label sets
A multi-label model maps an input to a binary indicator vector of length L, where L is the number of possible labels and each position states whether that label applies. The training data use the same encoding: a document with the tags sports and health becomes a vector with ones in two positions and zeros elsewhere.
In neural networks, the output layer applies an independent sigmoid to each label instead of a softmax across all of them. Each sigmoid yields a probability between 0 and 1 that does not compete with the others, so several outputs can be high simultaneously. Training typically minimizes binary cross-entropy summed over labels, which treats every label as its own yes-or-no decision.
The final label set comes from thresholding the scores. A global cutoff of 0.5 is the naive default, but production systems usually tune per-label thresholds on validation data, because rare labels rarely reach high scores. Some systems instead keep the top-k labels or predict the label count first and then select that many.
The table below summarizes the components that differ from a single-label setup.
| Component | Multi-label choice | Why it matters |
|---|---|---|
| Target encoding | Binary vector of length L | Represents any subset of labels |
| Output activation | Sigmoid per label | Scores do not have to sum to one |
| Loss function | Binary cross-entropy or focal loss | Each label is learned as its own decision |
| Decision rule | Per-label threshold or top-k | Converts scores into a label set |
| Evaluation | Hamming loss, micro and macro F1 | Rewards partially correct sets |
Problem transformation and algorithm adaptation methods
Classic approaches fall into two families. Problem transformation methods convert the multi-label task into one or more ordinary classification problems, while algorithm adaptation methods modify a learner so that it handles label sets directly.
Binary relevance trains one independent binary classifier per label, which is simple, parallel and scalable but ignores label correlations. Classifier chains link the binary models so that each one receives the previous predictions as extra features, which captures dependencies at the cost of sensitivity to chain order. Label powerset treats every distinct label combination as one class of a multi-class problem, which models correlations exactly but produces a combinatorial explosion of classes and cannot predict combinations unseen in training.
The following table compares the main methods on the properties practitioners weigh most often.
| Method | Label dependencies | Scalability | Main weakness |
|---|---|---|---|
| Binary relevance | Ignored | High, one model per label | Misses co-occurrence patterns |
| Classifier chains | Modeled sequentially | Moderate | Errors propagate along the chain |
| Label powerset | Modeled exactly | Low for many labels | Cannot output unseen combinations |
| RAkEL | Modeled within subsets | Moderate | Approximate, needs ensembling |
| ML-kNN | Captured from neighbors | Limited by data size | Slow at prediction time |
| Neural network with sigmoids | Learned in shared layers | High | Needs threshold calibration |
Algorithm adaptation includes ML-kNN, which estimates label probabilities from the labels of nearest neighbors, and multi-label decision trees that use a modified entropy across all labels. Modern deep models implicitly adapt the algorithm: a shared representation feeding many sigmoid outputs learns correlations through common hidden features.
How it differs from binary and multi-class classification
Multi-label classification generalizes both binary classification and multi-class classification. Binary tasks decide one yes-or-no question, multi-class tasks choose exactly one class from several, and multi-label tasks choose any subset of the available classes, including the empty set.
| Task type | Labels per example | Typical output layer | Example |
|---|---|---|---|
| Binary classification | One of two | Single sigmoid | Spam or not spam |
| Multi-class classification | Exactly one of many | Softmax | Digit recognition 0 to 9 |
| Multi-label classification | Zero to all of many | Sigmoid per label | Movie genres |
| Multi-output classification | One class per separate target | One softmax per target | Color and size of a garment |
A frequent design error is to model a multi-label problem with softmax. Softmax forces probabilities to sum to one, so two genuinely present labels must split the probability mass, and the model learns to suppress true co-occurrences. Multi-output classification is a related but distinct setting in which each target has its own class set.
Evaluating multi-label predictions
Evaluation is harder than in single-label tasks because a prediction can be partially correct. A model that outputs three of four true labels is better than one that outputs none, and the chosen metric decides how much credit it receives.
| Metric | What it measures | Behavior |
|---|---|---|
| Subset accuracy | Exact match of the whole label set | Very strict, low on large label spaces |
| Hamming loss | Share of wrong label decisions | Lenient, dominated by easy negatives |
| Micro-averaged F1 | F1 over all label decisions pooled | Weighted toward frequent labels |
| Macro-averaged F1 | Mean of per-label F1 scores | Exposes weak rare labels |
| Precision at k | Correct labels among top k | Suits ranking and tagging interfaces |
Reporting both micro and macro F1 is a sound default: a large gap between them signals that the model performs well on common labels and poorly on the long tail. Ranking metrics such as label ranking average precision are preferable when the system presents a sorted list of suggested tags to a human.
Limitations and common mistakes
Most real label spaces show severe class imbalance: a few labels appear in many examples and hundreds appear rarely. Plain resampling such as SMOTE is awkward here, because duplicating an example duplicates all of its labels. Practitioners rely instead on class-weighted or focal loss, iterative stratification for data splits and per-label thresholds.
- Using one global threshold leaves rare labels almost never predicted.
- Random train-test splits can place every example of a rare label in one partition; iterative stratification prevents this.
- Missing labels are common, because annotators tag what they notice, so an absent label is not always a true negative.
- Reporting only Hamming loss hides poor performance, since most label decisions are easy negatives.
- Extreme multi-label settings with tens of thousands of labels require specialized indexing, tree-based or embedding-based methods.
The idea in brief
Most sorting tasks ask a computer to put each item into exactly one box. Multi-label classification removes that restriction. A photo can be tagged beach, sunset and family at the same time, and a support email can be both billing and urgent.
Think of it as a checklist instead of a multiple-choice question. For every possible tag, the model asks a separate question: does this tag fit, yes or no. The answer is a list of all tags that apply, which may be long, short or even empty.
A practical scenario
A customer service team at a mid-sized software company receives about 4,000 support tickets per week and wants each one routed automatically. A ticket often touches several areas, such as a login failure that also involves a billing dispute, so the team frames the task as multi-label classification over 35 categories.
The data scientists fine-tune a text encoder with 35 sigmoid outputs and binary cross-entropy, split the data with iterative stratification and tune a separate threshold for each category on a validation set. A first model with a fixed 0.5 cutoff reached a macro F1 of about 0.48 because rare categories were rarely predicted; per-label thresholds raised it to roughly 0.63 while micro F1 stayed near 0.81.
In production, tickets are copied to every relevant queue. The outcome is fewer manual reassignments and faster first responses, and the team retrains monthly as new product features add labels.