Text classification is the task of assigning one or more predefined labels to a piece of text, such as a document, email, review, ticket or sentence. In speech & language AI it is the most widely deployed form of language understanding, because many business questions reduce to choosing a category: spam or not, which department, which product, how urgent.
A text classifier maps raw text to a label space defined in advance, which separates it from open-ended generation. Sentiment analysis, intent recognition and language identification are all specialized forms of text classification, each with its own label set and typical data.
How a text classifier turns words into labels
Every text classifier has two parts: a representation that turns text into numbers and a decision function that turns those numbers into label scores. Early systems used bag-of-words or TF-IDF vectors with linear models such as logistic regression or support vector machines. Modern systems use contextual embeddings from transformer encoders, which capture word order, negation and meaning in context.
In the standard fine-tuning approach, a pretrained encoder reads the text, a pooled vector (often the first token) feeds a small classification head, and a softmax or sigmoid produces probabilities per label. Training minimizes cross-entropy on labeled examples. Large language models add a third route: zero-shot or few-shot prompting, where the label names and a few examples are given in the prompt instead of training a dedicated model.
| Approach | Data needed | Strength | Weakness |
|---|---|---|---|
| TF-IDF with linear model | Hundreds to thousands of labels | Fast, cheap, transparent | Misses context and negation |
| Fine-tuned encoder | Roughly 1,000 or more examples | High accuracy, low latency | Needs labeled data and retraining |
| Embedding plus classifier | Small to medium sets | Reuses one embedding model | Slightly below full fine-tuning |
| LLM prompting | Label descriptions, few examples | No training, flexible labels | Higher cost, variable consistency |
Types of classification problems
The structure of the label space decides the model output and the metric. Binary classification chooses between two labels, multi-class classification picks exactly one of many, and multi-label classification allows several labels at once, as when a news article is both about politics and economics. Hierarchical classification uses a label tree, for example product category and subcategory.
| Problem type | Output layer | Example |
|---|---|---|
| Binary | One sigmoid or two-way softmax | Spam filtering |
| Multi-class | Softmax over all labels | Routing a ticket to one team |
| Multi-label | Independent sigmoid per label | Tagging articles with topics |
| Hierarchical | Per-level heads or flattened paths | Product taxonomy assignment |
| Ordinal | Ordered thresholds | Rating severity from 1 to 5 |
Choosing the wrong structure is a frequent design error: forcing a multi-label problem into multi-class discards valid labels and confuses training, because examples that legitimately belong to two classes appear contradictory.
Text classification compared with related language tasks
Several neighboring tasks look similar but differ in what the output refers to. Classification labels the whole text or sentence; named entity recognition labels spans inside the text; text summarization and question answering generate or extract new text rather than choosing from a fixed set.
| Task | Unit labeled | Output |
|---|---|---|
| Text classification | Whole text or sentence | Category from a fixed set |
| Sentiment analysis | Text, sentence or aspect | Polarity or emotion label |
| Intent recognition | User utterance | Intent plus optional slots |
| Named entity recognition | Token spans | Entity types per span |
| Text summarization | Whole document | Shorter generated text |
Building and evaluating a classifier
Quality depends first on the label guidelines and annotation consistency. If two trained annotators agree on only 80 percent of examples, no model can be reliably judged above that level. Teams therefore measure inter-annotator agreement, refine label definitions and merge categories that humans cannot separate.
Accuracy alone is misleading with imbalanced classes: a fraud filter that always answers no can exceed 99 percent accuracy. Use precision, recall and F1 per class, macro-averaged F1 when minority classes matter, and a confusion matrix to see which labels are mixed up.
| Situation | Recommended choice | Reason |
|---|---|---|
| No labeled data yet | LLM zero-shot, then collect labels | Fast start, builds a training set |
| High volume, low latency | Fine-tuned small encoder | Milliseconds per item at low cost |
| Labels change often | Prompting or embedding classifier | Avoids repeated full retraining |
| Audit requirements | Linear model or calibrated encoder | Explainable features and scores |
Pitfalls in production
Deployed classifiers degrade through data drift: new products, new slang or a changed form layout shift the input distribution. Monitoring the distribution of predicted labels and periodically labeling a fresh sample reveals drift before users notice it.
- Uncalibrated scores: a probability of 0.9 may not mean 90 percent correct; calibrate before using thresholds.
- Label leakage: signatures, ticket prefixes or templates can reveal the label and inflate test results.
- Bias in training data: classifiers can learn associations with dialect or names; test performance across groups.
- Missing reject option: route low-confidence items to people instead of forcing a label.
Long inputs create a separate problem. Many encoders accept only 512 tokens, so long contracts or reports must be truncated, split into chunks whose predictions are aggregated, or handled by a long-context model. Truncation silently drops evidence that appears late in a document, which is why chunking with a max or mean aggregation rule is usually safer for multi-page texts.
Finally, a classifier is only as useful as the process around it. Each label should map to a concrete action, such as a queue, a workflow or a filter, and the cost of each error type should be agreed in advance. A misrouted billing question is an annoyance; a missed safety complaint may be serious, so thresholds per label should reflect those costs rather than a single global setting.
In simple terms
Think of a mailroom clerk who reads each incoming letter and drops it into the right tray: invoices, complaints, job applications. Text classification is that clerk in software form. It reads a message and puts it into one of the trays you defined in advance.
The system learns which tray fits by studying many letters that people have already sorted. It does not write anything new; it only chooses the best label, and good systems also say how sure they are.
A practical example
An online retailer receives about 20,000 customer emails per week, and agents used to sort them by hand into returns, delivery problems, billing, product questions and other. The support team labels 6,000 historical emails with clear guidelines and fine-tunes a compact transformer encoder on them.
On a held-out set the classifier reaches a macro F1 of around 0.9, with most confusion between delivery and returns. Emails above a confidence threshold go straight to the matching queue, while uncertain ones go to a human triage step. The team reviews a weekly sample, and first-response time drops because agents start on messages that already sit in the right queue.