AI Is Driving Greater Accuracy in Advanced Speech Recognition

“I’m sorry, I didn’t quite get that.”

Most of us have had an experience like this with speech-recognition solutions deployed for customer support or other public-facing applications. In many cases, this has led to low public opinion of the technology, which has often made us wonder if we’re speaking properly. (There’s nothing like being gaslit by a machine.)

The good news is that there is evidence that AI is driving greater accuracy in advanced speech recognition (ASR) technology. A new report from 3Play Media, a media accessibility company, found that the accuracy of ASR technology has improved measurably since the company’s last evaluation in 2022. As ASR improves, it's important to understand which engine is best for different use cases. Some nuances to consider include performance on different error types, transcription styles, formatting, and industry-specific content.

Accuracy is the key component in captioning for several reasons, most importantly ensuring that individuals who are deaf or hard of hearing and rely on captions as an accommodation receive information that fully depicts the original content. For captions to be accessible and legally compliant, they need to be 99 percent accurate, the industry requirement for accessibility. While there was improvement across industry leaders, the study found that even the best engines performed well below 99 percent accuracy, indicating a continued need for human revision.

The report, titled “The 2023 State of ASR,” measured accuracy against two measurements, Word Error Rate (WER) and Formatted Error Rate (FER). While WER is used as the standard measure of transcription accuracy, FER considers formatting, sound effects, grammar, and punctuation and is a better representation of the experienced accuracy of captioning. Accuracy in FER is harder to achieve, and even the best-tested engines were only 82 percent accurate, whereas the best-tested engines in WER were 93 percent accurate.

“The advances in AI we’ve seen across industries have also had an impact on ASR,” said Chris Antunes, co-CEO and cofounder of 3Play Media. “Longtime industry leader Speechmatics and newer entrants AssemblyAI and Whisper performed at the top of the pack, with each excelling in different areas. This proves that not all engines are created equal -- the training material and models matter -  and that there is room at the top for multiple engines to specialize in different use cases.”