Imagine stepping off a plane anywhere in the world, slipping on a pair of smart glasses, and instantly understanding everything being said around you.
McGill sociology professor Skyler Wang is part of an international team working alongside researchers from Meta to develop a universal translator. The team envisions that within five years, commercial prototypes will exist that can provide near-simultaneous, offline translation.
Existing machine translation models and devices (such as earbuds, handheld translators, and smart glasses) currently support only a handful of widely-spoken languages, including English, Spanish, French and Mandarin.
More than 7,000 languages are spoken worldwide; the team’s goal is to dramatically expand machine translation’s linguistic reach. In their latest Nature article, they introduce SeamlessM4T (Massively Multilingual & Multimodal Machine Translation), a single state-of-the-art AI model that supports text-to-text translation in approximately 200 languages and speech-to-speech translation in around 100. While the model covers a significant portion of the global population’s translation needs, it still excludes many of the world’s languages. Efforts are underway to improve both the quality and language coverage of the team’s contributions.
To build the foundational models on which the systems rely, the researchers used a variety of publicly available and curated data sources. They also partnered with local communities and linguistics experts to gather additional translations, doing their best to ensure that the process was ethical and inclusive.
“We were very conscious about not wanting to be overbearing,” Wang explained, citing his effort to work with many low-resource language speakers to ascertain their needs through a community-centric approach. This philosophy led to the creation of the Open Language Data Initiative, an open-source repository where contributors can add translations of over 6,000 sentences extracted from English Wikipedia to help train and evaluate future machine translation models.
Wang, a sociologist with an interest in human-machine interactions and social-centred AI, emphasizes that human involvement remains crucial.
“The ethical side of this work is incredibly complex,” he said. “It’s not just about the speed and quality of this technology. It’s about respecting the people and cultures behind the languages and not engaging in a new form of digital colonialism.”
Wang said that, given the immense resources required to move the project forward, the collaboration between academia and big tech is essential to bringing the project to fruition. The team’s open-source approach enables others to build upon its foundational work. SeamlessM4T has already been used in Switzerland, where it has helped to bridge communication gaps between aid workers and displaced individuals.
The implications are profound. Wang said these tools could help preserve endangered languages, foster cross-cultural understanding and democratize access to knowledge.
“Translation isn’t just about access,” he noted. “It’s about preserving cultural heritage and enabling global knowledge exchange.”
Despite ongoing challenges, such as minimizing low-quality translations and developing user-friendly devices to support broad adoption, the project has already garnered international recognition. Notably, Time magazine named the SeamlessM4T project one of the top 200 inventions of 2023.
“It’s not perfect,” Wang said, “but the feedback from communities has been encouraging. We’ve contributed something that could truly have an impact on how the world communicates.”