Can AI Translate Languages the Internet Has Barely Learned?

The hardest problem in machine translation may not be building a larger model, but teaching it a language for which the digital world offers little usable data.

Imagine asking two students to learn translation. One receives a library filled with books, subtitles, news reports and millions of professionally translated sentences. The other receives a dictionary, a handful of documents and fragments gathered from different sources.

The difference is not intelligence. It is opportunity.

Artificial intelligence faces a similar imbalance. Translation has improved rapidly for languages richly represented online, but far less evenly for those with little digitised text or bilingual material. A peer-reviewed survey of low-resource machine translation notes that roughly 7,000 languages are spoken worldwide, yet almost all language pairs lack significant training resources.

These languages are often called “low-resource”, although “under-resourced” better captures the problem. The term does not mean that a language is simple, unimportant or spoken by very few people. It means that the machine-readable material needed to build and evaluate language technology is scarce.

AI learns from examples, not dictionaries alone

A dictionary can tell a system that two words are related. Translation, however, involves grammar, context, idiom and word order. The same word may require a different translation in a government notice, a casual conversation or a medical document. A useful system must learn these relationships from sentences and longer passages.

For well-resourced language pairs, developers may have access to large “parallel corpora”: collections in which the same material appears in both languages. These aligned examples show a model how people express the same meaning on each side.

For an under-resourced language, the problem becomes circular. Too few digital translations make it difficult to train a reliable model; without a reliable model, translating and digitising more material remains slow and expensive.

When English Has 8,800 Times More Data

The imbalance is visible across the public internet. In the CC-MAIN-2026-30 release of Common Crawl, one of the largest open collections of webpages used in language-model research, English accounts for about 40.6% of pages whose primary language could be identified. Assamese accounts for just 0.0046% — a difference of roughly 8,800 to one.

This is a measure of online visibility, not the training set of any particular AI model. But it shows how much easier it is to collect text at scale in one language than in another.

The gap is similarly stark in parallel data, which is especially valuable for machine translation. The same survey found about 280 million English–French sentence pairs in the OPUS collection, compared with 700,000 for English–Myanmar and 35,000 for English–Fon, a language spoken by millions in West Africa. On that measure, the English–French resource was about 400 times larger than English–Myanmar and 8,000 times larger than English–Fon.

Scarcity is not only about volume. The data that do exist may come largely from religious publications, government documents or a small number of educational projects. A model trained on such material may learn formal sentence structures while struggling with everyday speech, contemporary terminology, regional expressions or specialised subjects such as healthcare and technology.

How researchers work around the data shortage

One approach is multilingual transfer: training a system across many languages so that patterns learned from data-rich languages can help it handle those with fewer examples. Another is back-translation, in which a model creates synthetic sentence pairs from monolingual text to supplement scarce human-translated material.

Neither technique creates knowledge from nothing. Synthetic text can reproduce errors, while inconsistent spelling and narrow subject coverage can limit apparent gains. Native-speaker participation therefore remains essential to determine whether a translation is both accurate and natural in context.

A recent test in Northeast Indian languages

WMT, the Conference on Machine Translation, is one of the field’s most established benchmark series, bringing together research institutions and technology teams to evaluate translation systems under standardised conditions.

The WMT26 Low-Resource Indic Language Translation shared task divided its languages according to the amount of available training data. Its second category — Bodo, Kokborok, Karbi, Nagamese and Tagin — was explicitly labelled “Very Limited Training Data”.

AI company VoxNexus entered all five Category 2 language-to-English tracks. In the published results, its systems produced the top reported scores in four: Kokborok, Karbi, Nagamese and Tagin, leading all listed systems on each of the six automatic evaluation metrics used.

The result does not mean that machine translation for these languages is solved. It does, however, offer evidence that meaningful technical progress is possible even when conventional training material is unusually scarce.

The real test begins outside the leaderboard

A benchmark is only a starting point. Real-world translation must work across unfamiliar subjects, regional expressions and imperfect everyday language. Many challenges remain, from building reliable datasets to ensuring that translations are accurate, natural and responsibly developed.

But the potential value is substantial. Better AI for under-resourced languages could widen access to education, public information, search and digital services. **The true measure of progress will not be how many languages AI can translate, but how many people no longer have to leave their language behind to participate in the digital world.**

Note to readers: This article is part of Mint’s paid consumer connect Initiative. Mint assumes no editorial involvement or responsibility for errors, omissions, or content accuracy.

Want to get your story featured as above? click here!