How to calculate word error rate (WER)
Word error rate (WER) is the standard accuracy metric for transcription. The formula is:
WER = (S + D + I) / N
where S = substitutions, D = deletions, I = insertions, and N = the number of words in the reference (correct) transcript. The three error counts come from aligning the machine output to the reference using the minimum word-level edit (Levenshtein) distance. Lower is better.
A worked example
- Reference (10 words): “the quick brown fox jumps over the lazy dog today”
- Output: “the quick brown fox jumped over lazy dog now”
Aligning the two:
- “jumps” -> “jumped” = 1 substitution
- “the” (before “lazy”) missing = 1 deletion
- “today” -> “now” = 1 substitution
- no extra words added = 0 insertions
So S = 2, D = 1, I = 0, N = 10:
WER = (2 + 1 + 0) / 10 = 0.30 = 30%
The gotchas that change the number
WER is deceptively sensitive to how you prepare the text before comparing:
- Casing and punctuation — “Hello.” vs “hello” counts as an error unless you normalize.
- Numbers and contractions — “twenty-five” vs “25”, “do not” vs “don’t”.
- Whitespace and filler words — “um”, “uh” inflate counts if the reference omits them.
Always normalize both transcripts the same way (lowercase, strip punctuation, expand or collapse numbers consistently) before scoring — otherwise you’re measuring formatting, not recognition accuracy.
Computing it in practice
In Python, the open-source jiwer library handles normalization and returns WER from a reference and a hypothesis string in a couple of lines. For a single comparison by hand, count S, D and I from the alignment and divide by N.
Remember a published WER is only a guide: it depends on the audio, the speaker and the domain. The honest way to compare tools is to run your own audio through them — see the best on-prem transcription ranking for tools you can benchmark privately on your own hardware.
Frequently asked questions
What is the formula for word error rate?
WER = (S + D + I) / N, where S is substituted words, D is deleted words, I is inserted words, and N is the number of words in the reference transcript. The three error counts come from the minimum edit (Levenshtein) distance between the reference and the output.
What is a good WER?
On clean audio, modern Whisper-class models reach low single-digit WER (around 3-5%). Anything under ~10% is usually workable; noisy audio, heavy accents or specialist vocabulary push it higher. There's no universal threshold — it depends on the use case.
Can WER be greater than 100%?
Yes. Because insertions are counted, a system that outputs many extra words can produce more errors than there are reference words, giving a WER above 100%. It's uncommon but valid.
How do I calculate WER in Python?
Use the open-source jiwer library — it normalizes the text and returns WER from a reference and a hypothesis string. Doing it by hand means computing the Levenshtein edit distance at the word level and dividing by the reference word count.