There are around 7,000 languages in the world, but existing speech recognition models only cover approximately 100 languages comprehensively. This is because these kinds of models tend to require huge amounts of labeled training data, which is only available for a small number of languages, including English, Spanish, and Chinese.
Meta researchers got around this problem by retraining an existing AI model developed by the company in 2020 that is able to learn speech patterns from audio without requiring large amounts of labeled data, such as transcripts.
They trained it on two new datasets: one that contains audio recordings of the New Testament Bible and its corresponding text taken from the internet in 1,107 languages, and another containing unlabeled New Testament audio recordings in 3,809 languages. The team processed the speech audio and the text data to improve its quality before running an algorithm designed to align audio recordings with accompanying text. They then repeated this process with a second algorithm trained on the newly-aligned data. This method meant that the researchers were able to teach the algorithm to learn a new language more easily, even without the accompanying text.
“We can use what that model learned to then quickly build speech systems with very, very little data,” says Michael Auli, a research scientist at Meta who worked on the project.
“For English, we have lots and lots of good data sets, and we have that for a few more languages, but we just don’t have that for languages that are spoken by, say, 1,000 people.”
The researchers say their models can converse in over 1,000 languages but recognize more than 4,000.
They compared the models’ accuracy to those from rival companies including OpenAI Whisper, and claim theirs achieved half the error rate, despite covering 11 times more languages.
However, the team warns the model is still at risk of mistranscribing certain words or phrases, which could result in inaccurate or potentially offensive labels. They also acknowledged that the language produced by their speech recognition models contained more biased words compared to other models, albeit at a small rate—0.7% more.
While the scope of the research is impressive, the use of religious texts to train AI models can be controversial, says Chris Emezue, a researcher at Masakhane, an organization working on natural-language processing for African languages, who was not involved in the project.
“The Bible has a lot of bias and misrepresentations,” he says.