Named entity recognition (NER) is the natural language processing task of locating mentions of named entities in text and classifying them into predefined types, such as person, organization, location, date or monetary amount. In speech and language artificial intelligence, NER turns unstructured text into structured, searchable data and is a core step in information extraction pipelines.
An NER system reads “The board of Nordwind Logistics met in Rotterdam on 4 March” and returns three labeled spans: an organization, a location and a date. The task has two parts that must both be right: finding the exact boundaries of each mention and assigning the correct entity type.
How named entity recognition works
Most NER systems treat the task as sequence labeling: the text is split into tokens, and the model assigns each token a tag that marks whether it begins an entity, continues one or lies outside any entity. The most common scheme is BIO tagging, where B-ORG marks the first token of an organization, I-ORG the following tokens and O everything else.
Classic systems used conditional random fields over hand-crafted features such as capitalization, word shape and gazetteer lists. Modern systems use a transformer encoder that produces a contextual embedding for every token, followed by a classification layer, sometimes with a CRF on top to enforce valid tag sequences, for example that I-PER never follows B-ORG.
A newer approach uses generative language models that are prompted to list entities in a structured format such as JSON. This makes it easy to add new entity types through instructions alone, but it requires output validation, because the model may return spans that do not appear verbatim in the source text.
Entity types and tagging schemes
The set of entity types, called the label schema, is chosen for the application. General-purpose models usually follow a standard schema, while domain systems define their own types.
| Entity type | Typical tag | Example mention | Common in |
|---|---|---|---|
| Person | PER | Ada Lovelace | News, legal, customer service |
| Organization | ORG | World Health Organization | Finance, news |
| Location | LOC or GPE | Rotterdam | Logistics, travel |
| Date and time | DATE, TIME | 4 March 2025 | Contracts, scheduling |
| Money and quantity | MONEY, QUANTITY | 12,500 euros | Invoices, finance |
| Domain-specific type | DRUG, GENE, PRODUCT | metformin | Medicine, biology, retail |
Tagging schemes differ in detail. BIOES adds tags for the end of an entity and for single-token entities, which can slightly help boundary detection. Nested NER handles cases such as “Bank of England” inside “Governor of the Bank of England”, which flat BIO tagging cannot represent.
Approaches compared
NER approaches trade accuracy, adaptability, cost and control against each other. The right choice depends on how stable the entity types are and how much labeled data is available.
| Approach | Data needed | Strength | Weakness |
|---|---|---|---|
| Rules and gazetteers | Lists and patterns | Precise, explainable | Low recall on new names |
| Statistical models with CRF | Thousands of labeled sentences | Fast, small | Feature engineering, weaker context |
| Fine-tuned transformer encoder | Hundreds to thousands of labeled sentences | High accuracy, fast inference | Fixed label set |
| Prompted generative model | A few examples or none | New types without training | Higher cost, boundary errors |
| Hybrid pipeline | Mixed | Rules for formats, model for names | More components to maintain |
In production, a fine-tuned encoder model remains the usual choice for high-volume, stable schemas because it is fast, cheap and consistent. Generative models are often used to bootstrap training data or to cover rare types, and their output is then used to train a smaller encoder.
How NER is evaluated
NER is evaluated at the entity level, not the token level. A predicted entity counts as correct only when both its span and its type exactly match the reference annotation in strict evaluation.
| Evaluation mode | Counts as correct when | Use |
|---|---|---|
| Strict match | Exact span and correct type | Standard benchmark reporting |
| Partial match | Overlapping span and correct type | Applications tolerant of boundary errors |
| Type-only match | Correct type, any overlap | Analysis of classification errors |
| Span-only match | Exact span, any type | Analysis of boundary errors |
Results are reported as precision, recall and F1 score, usually micro-averaged across entity types, with per-type scores revealing weak categories. Scores on well-studied English news benchmarks are high, but performance drops noticeably on new domains, informal text and other languages, so in-domain test sets are essential.
Relation to neighboring language tasks
NER is one step in a chain of language understanding tasks. It finds and types mentions, while neighboring tasks interpret them further.
- Part-of-speech tagging labels the grammatical role of every word; NER labels only spans that name things.
- Entity linking connects a recognized mention to a specific record, deciding which of several people named Smith is meant.
- Relation extraction finds links between entities, such as which company acquired which.
- Intent recognition classifies what a user wants, often using NER output as slot values.
- Text classification assigns labels to whole documents rather than spans.
In speech applications, NER runs on speech-to-text transcripts, where missing capitalization and recognition errors make the task noticeably harder, so models trained on transcribed speech perform better than models trained on written text.
Common challenges and mistakes
The hardest NER problems come from ambiguity and domain mismatch. “Jordan” can be a person, a country or a brand, and only context decides; a model trained on news may not recognize product codes or drug names at all.
Frequent mistakes include inconsistent annotation guidelines, which cap achievable accuracy, ignoring nested and discontinuous entities, and evaluating only on clean text when production input is noisy. When NER is used to redact personal data, recall matters far more than precision, because a single missed name can expose a person; such systems should be tuned and tested with that asymmetry in mind.
In plain words
Named entity recognition is like giving a computer a set of colored highlighters. As it reads, it marks names of people in one color, companies in another, places and dates in others. The result is a document where the important names stand out and can be collected automatically.
The tricky part is that the same word can mean different things. “Paris” might be a city or a person, so the system has to use the surrounding words to decide, much as a careful human reader would.
A real-world example
A hospital’s research department wants to share thousands of discharge letters with a study team without revealing patient identities. It deploys an NER model fine-tuned on about two thousand annotated clinical letters to find patient names, dates of birth, addresses, phone numbers and the names of treating doctors.
Every detected entity is replaced with a placeholder of its type, such as a generic name tag, while drug names and diagnoses are left intact for research. Because a missed name is far more harmful than an extra redaction, the team sets the model to favor recall over precision and adds regular expressions for phone numbers and record IDs. A manual review of a random sample confirms that very few identifiers slip through, and the letters are released for the study.