Speech & Audio · entry 03/06 · 2 min read

Whose voice it hears

Speech recognition does not fail uniformly across speakers, and the headline accuracy number is exactly the wrong shape for noticing — where the gaps come from and how to measure them.

The gap is measured, not alleged

Peer-reviewed audits of commercial speech recognition have repeatedly found substantial differences in word error rate across speaker groups — by dialect and race, by first language, by age, and by whether the speaker has a speech difference such as a stutter or dysarthria. The best-documented result found error rates roughly twice as high for Black American speakers as for white speakers of the same language, across five major systems at once, on carefully matched interview audio. The systems have improved since; the shape of the finding has not gone away.

Four places it comes from

Acoustic data: training corpora over-represent the speakers who were easiest and cheapest to record, so the model has heard some voices far more than others. The language prior: a recognizer does not only match sounds, it predicts likely word sequences — and a prior trained mostly on standardized written English will quietly "correct" a grammatical dialect into something it expected. The lexicon: names, place names, and code-switching land outside the vocabulary the system was tuned on. The evaluation set: if the test audio inherits the training data's composition, the disparity is real and simply not visible in the number anyone reports.

Why it lands harder than a number suggests

Voice increasingly sits in front of things people need: phone systems that route benefit claims, clinical dictation, automatic captions in a classroom, voice controls in a car. A system at 95% average accuracy is a working product for some users and an obstacle course for others, and the ones it fails have no way to appeal — they are simply told, over and over, that it did not catch that. This is the same failure the fairness entry describes, arriving through audio instead of a score.

Failure mode

Reporting one WER. An average over a convenience sample is the metric most likely to hide exactly the thing you should know, and "our accuracy is 95%" is not a claim about any particular user. Break the number out by whatever groups your product actually serves, collect evaluation audio that represents them rather than the audio that was easy to get, and treat a gap as a defect with an owner. The fix is almost never telling people to speak more clearly.