Word error rate (WER) measures speech-recognition accuracy by counting word-level substitutions, deletions, and insertions against a reference transcript, then expressing the errors as a proportion of the reference words. section 4.4.2
WER makes performance differences concrete, but the result depends strongly on the test conditions. DS2 performs better than human listeners on several read-speech sets, while noisy CHiME speech produces a much higher error rate. section 4.4.2
The reported comparisons show how WER changes across datasets:
- WSJ eval’92: DS2 3.60 WER versus human 5.03. section 4.4.2
- LibriSpeech test-clean: DS2 5.33 versus human 5.83; test-other: DS2 13.25 versus human 12.69. section 4.4.2
- CHiME: DS2 21.79 versus human 11.84, illustrating the cost of noisy environments. section 4.4.2
Table 4.1 WER for DS2 and human-level performance on read speech. CHiME data were collected in noisy environments, including a bus, a café, a street, and a pedestrian area.
|
Test set
|
DS2
|
Human
|
|---|---|---|
|
WSJ eval’92
|
3.60
|
5.03
|
|
WSJ eval’93
|
4.98
|
8.08
|
|
LibriSpeech test-clean
|
5.33
|
5.83
|
|
LibriSpeech test-other
|
13.25
|
12.69
|
|
VoxForge (American-Canadian)
|
7.55
|
4.85
|
|
CHiME
|
21.79
|
11.84
|
Use the comparison table to inspect DS2 and human WER across read-speech datasets, including noisy CHiME recordings. section 4.4.2
