Word error rate

Also known as: WER

On this page 7

Word error rate (WER) is the standard metric for speech recognition accuracy: the number of word substitutions, deletions and insertions needed to turn a system transcript into a human reference transcript, divided by the number of words in the reference. In speech & language AI, WER is the headline figure for comparing speech-to-text engines, acoustic models and end-to-end recognizers, and a lower value means a better transcript.

Because WER is computed from a minimum edit distance at the word level, it is simple, language-agnostic in form and easy to reproduce. Its simplicity is also its weakness: every word counts equally, so a misheard drug name and a missing filler word cost exactly the same.

How word error rate is calculated

WER is calculated by aligning the hypothesis with the reference using the Levenshtein algorithm on word tokens, then counting the three error types the cheapest alignment contains. The formula is WER = (S + D + I) / N, where S is substitutions, D is deletions, I is insertions and N is the word count of the reference.

The alignment uses dynamic programming: each cell of a matrix stores the lowest cost of matching a prefix of the reference with a prefix of the hypothesis. When several alignments share the same minimum cost, scoring tools pick one by a fixed tie-breaking rule, which can shift errors between categories but never changes the total.

The following table shows the three error types with a short illustration for the reference sentence “book a table for two”.

Error typeWhat happensHypothesis exampleCount
Substitution (S)A reference word is replaced by another wordbook a cable for two1 S
Deletion (D)A reference word is missingbook table for two1 D
Insertion (I)An extra word appearsbook a table for the two1 I
Correct matchWords align exactlybook a table for two0
Error types counted by word error rate

Because insertions are added to the numerator but not to N, WER can exceed 100%. A system that outputs a long stream of hallucinated words for a short utterance can score 150% or more, which is a known surprise for newcomers who read WER as a percentage of wrong words.

Normalization before scoring

Text normalization decides what counts as the same word, and it often moves WER more than a model change does. Reference and hypothesis must pass through identical normalization rules before alignment, otherwise the metric measures formatting differences instead of recognition errors.

Typical normalization steps and their effect are summarized below.

Normalization stepExampleTypical effect on WER
Lowercasing“Paris” and “paris” matchRemoves case mismatches
Punctuation removal“two.” becomes “two”Removes punctuation-only errors
Number expansion“25” versus “twenty five”Can change WER by several points
Spelling variant unification“e-mail” versus “email”Avoids penalizing valid variants
Filler word removal“uh”, “um” droppedLowers WER on conversational speech
Common normalization choices and their impact

A frequent mistake is to compare a published WER with an in-house figure without checking the normalizer. Two numbers are comparable only when the test set, the normalization script and the scoring tool are the same; scoring tools such as NIST SCTK and the open-source jiwer library are common reference implementations.

Typical values and what they depend on

There is no universal good WER, because the figure depends on audio quality, speaking style, vocabulary and accent coverage. The same engine can score in the low single digits on read speech and above 20% on noisy, overlapping conversation.

The table gives broad, approximate ranges seen with modern recognizers; actual values vary widely by language and dataset.

Audio conditionApproximate WER rangeMain difficulty
Clean read speech2-6%Rare words and names
Broadcast news5-10%Music beds and fast delivery
Phone calls10-20%Narrowband audio and disfluencies
Meetings with far-field microphones15-35%Reverberation and overlapping speakers
Low-resource languages20-50% or moreScarce training data
Approximate WER ranges by audio condition

Human transcribers also disagree with each other. On conversational telephone speech, inter-annotator WER of roughly 4-6% has been reported, which is why claims of “human parity” depend heavily on how the human baseline was produced.

WER is one member of a family of edit-distance metrics, and the right choice depends on the language and the downstream task. For languages without spaces between words, or for tasks where meaning matters more than exact wording, other measures are more informative.

MetricUnit of comparisonBest suited for
Character error rate (CER)CharactersChinese, Japanese, Thai and OCR
Sentence error rate (SER)Whole utterancesShort voice commands
Match error rate (MER)Words, bounded to 0-100%Comparing systems with many insertions
Entity or keyword error rateNamed entities or key termsMedical, legal and contact-center use
Semantic distanceMeaning similaritySpoken language understanding pipelines
Metrics related to word error rate

WER also differs from speaker diarization error rate, which measures who spoke when rather than what was said. Systems that combine transcription and diarization often report a concatenated minimum-permutation WER so that word errors and speaker attribution errors are scored together.

Limitations and common mistakes

The central limitation is that WER weights all words equally. A transcript of a prescription that turns “fifteen milligrams” into “fifty milligrams” has one substitution, the same cost as dropping “the”, yet the consequence is completely different.

  • Small test sets produce unstable figures; a difference of 0.3 points on a few hundred utterances is usually within noise.
  • Averaging per-utterance WER gives a different result from pooling all errors over all reference words; the pooled figure is the standard.
  • Reference errors in the ground truth inflate WER and can penalize a system that was actually correct.
  • Optimizing only for WER can hide regressions in punctuation, capitalization and readability, which WER usually ignores after normalization.
  • Large generative recognizers can hallucinate fluent text during silence, which shows up as bursts of insertions.

Good practice is to report WER together with confidence intervals, test set details and the normalizer, and to add a task-specific measure such as entity accuracy whenever particular words carry the business or safety risk.

A plain-language summary

Word error rate is a score for how many words a transcription got wrong. Imagine a teacher marking a dictation: every wrong word, missing word and extra word gets a mark, and the total is divided by the number of words in the correct text.

A score of 5% means roughly one mistake in every twenty words. Lower is better, but the score treats every word the same, so a wrong name or number counts no more than a wrong “a” or “the”.

An example from practice

A hospital evaluates two speech-to-text services for dictated radiology reports. The team prepares 200 dictations with verified reference transcripts, about 38,000 words in total, and runs both services through the same normalization script that expands numbers and removes punctuation.

Service A scores 6.1% WER and Service B scores 6.8% WER, a gap the team checks with bootstrap resampling and finds statistically meaningful. However, a second count of errors on drug names, dosages and laterality terms such as “left” and “right” shows Service B makes fewer of these critical mistakes.

The hospital selects Service B with a custom vocabulary, because the clinically important error rate matters more than the overall WER. The final evaluation report presents both numbers side by side so that later model updates are judged on the same two criteria.