Background – AI, LLMs and language: According to a 2021 study led by the Australian National University, at least 1,500 languages worldwide are at risk of disappearing by the end of this century. A Hong Kong-based company hopes to use large language models (LLMs) powered by artificial intelligence (AI) to prevent this.
Preserving Cantonese has been a challenge. Limited learning resources, a general lack of standardisation and Mandarin’s dominance have all contributed to a declining number of young learners.
The language faces an uncertain future. But Hong Kong-based deep-tech company Votee AI hopes to change things before it’s too late. The company hopes to use large language models (LLMs) powered by artificial intelligence (AI) to preserve languages that are often overlooked by tech giants, such as Cantonese.
Leo Ma, Votee’s chief scientist for the Asia-Pacific, said AI could create living records of languages.
“[But] while mainstream AI models excel in English, they remain functionally illiterate for 99 per cent of the world’s languages,” Ma said, adding that this limits access to AI for billions of non-English speakers.
‘Low-resource language’
Unlike Mandarin, English or Spanish, which have vast troves of digital text for AI training, Cantonese lacks written text that accurately reflects how it is spoken.
This makes Cantonese a “low-resource language” because there is little written text available for training, Ma said.
Votee AI, which says it is the first company to open-source a Cantonese LLM, went through the demanding challenges of recording the language spoken by 85 million people worldwide.
“Open source is a critical path to ensuring linguistic diversity survives in the age of AI,” Ma said.
To collect training data, the team built a digital dictionary. They gathered data from the internet, including social media and forums, and from partners such as universities, then annotated the cultural context of words.
“Words can have many meanings in different contexts,” Ma said. “Every word, corresponding to a specific context, will be represented by its own independent item in the data set we create.”
For example, in Cantonese, the word for “water” is often used to refer to money. According to Ma, this level of deep understanding is crucial: models trained on other languages cannot capture the unique idioms and cultural nuances of Cantonese speakers through translation.
“This is not merely a matter of inaccurate translation; it is a problem of cultural misrepresentation,” he said.
Linguistic precision is also important for the real-world application of LLMs. For models deployed in sectors such as finance, medicine, law and government, having “half-fluent foreign interns” with 50 per cent accuracy represented a liability, Ma said.
A new AI-powered process
To ensure models reflected rich cultural nuances, a Votee AI researcher created a benchmark with peers. It aimed to elevate the language proficiency, cultural knowledge, reasoning and problem-solving of Cantonese LLMs in the context of Hong Kong.
Ma said the initial data cleansing for the Cantonese data set took the team four to five months, but the company had built a system using autonomous agents that shortened the process to two weeks.
“Training a new model used to be a months-long process involving data cleansing, persona creation and data synthesis,” Ma explained. “We needed data representing various speech patterns – for example, how young trend hoppers speak, the language used by lawyers or the way celebrities talk.”
The new AI-powered process could be replicated for other low-resource languages in Southeast Asia and Africa, Ma said, adding that the company had set up an AI research lab in Toronto, Canada, and regional hubs in Kuala Lumpur, Malaysia, and Ho Chi Minh City, Vietnam.
“This platform is not just for one or two languages; its goal is to dramatically lower the technical and cost barriers to building high-quality, native-language models,” he said.
Importance of preserving languages
Ma pointed to the centuries-long inability to understand and pronounce Egyptian hieroglyphs. The meaning of the language remained a mystery until the Rosetta Stone was discovered in 1799.
It finally allowed scholars to translate Egyptian hieroglyphs thanks to its inscriptions, which showed the same content in three scripts, including Ancient Greek. It served as the translation guide.
“When a language disappears – how can we know its pronunciation and the true meaning of its words? No one can be 100 per cent certain, so parts of that culture are likely lost,” Ma said. “To understand the true nature of a culture, we must study its language and texts.”
Ma said humans remained central to ensuring the performance of new models.
“The passing down of languages in communities is incredibly important,” Ma said. “If there is nobody left to speak a language, the language is already lost. We believe Al’s role is not only to create the future but also to safeguard the past.”
