Anyone who has used a translation app across several languages has noticed the quality is wildly uneven. French, Spanish and German come back fluent. Some languages with tens of millions of speakers come back mangled or not at all.
The reason is not that some languages are harder in some abstract sense. It is arithmetic about training data.
Machine translation learns from parallel text
Modern translation systems are trained on parallel text — the same content in two languages, aligned. The more of it exists, the better the system gets.
For a handful of languages, that material is abundant. European Union institutions have produced decades of documents translated across all official languages. The United Nations does the same. Add commercially subtitled film and television, translated news, and technical documentation, and French-English has an enormous corpus behind it.
For most of the world's languages, almost none of that exists. There is no parliamentary record in Hakha Chin, no subtitled back-catalogue in Cheyenne, no treaty archive in Mizo. A system has very little to learn from, and output quality follows directly.
Speaker count is the wrong predictor
This is the counter-intuitive part, and it is worth stating clearly with examples.
| Language | Rough speakers | Digital resources |
|---|---|---|
| Icelandic | ~350,000 | relatively good — wealthy country, strong state investment in language technology |
| Khmer | ~17 million | thin relative to speaker count |
| Mizo | several hundred thousand | thin, despite official state status in India |
| Lao | ~4 million plus related varieties | thin |
Icelandic has a fiftieth of Khmer's speakers and considerably better digital coverage, because resourcing follows economic and institutional attention rather than population. That is the whole mechanism. A language becomes well-served when somebody with money has a reason to serve it.
Under-resourced is not endangered. These get conflated constantly and the distinction matters. Mizo is used in education, administration and broadcasting in Mizoram and is being passed to children normally — it is a healthy language that happens to be thinly represented in software. Cheyenne, by contrast, is severely endangered, with most fluent speakers elderly. Both are under-resourced; only one is at risk of disappearing, and they need different things.
The features that make it harder still
Beyond data volume, some structural properties compound the problem.
No word spaces. Thai, Lao, Khmer and traditionally Burmese are written without spaces between words. Before translating, a system must segment the text, and segmentation errors propagate into everything downstream.
Rich morphology. A Finnish noun appears in fifteen cases across singular and plural; a polysynthetic language like Cheyenne or Ojibwe builds single words carrying what English needs a clause for. Every additional form is another thing a system needs examples of, so the same amount of text teaches it less.
Unsettled orthography. Languages whose writing systems are recent, or which are written differently by different communities, split what little data exists across competing spellings. Several of the languages in our catalogue have this property, and it is why a word spelled one way in one source appears differently in another.
Where a compiled dictionary wins
For a well-resourced language, a translation app is often more convenient than a dictionary — it handles whole sentences, and its single answer is usually right.
For an under-resourced language the balance shifts, for a specific reason. A dictionary entry was written by a person who knew the language. A machine translation of a language with little training data is a guess, and it will present that guess with exactly the same confidence as a good answer. That is the dangerous property: there is no signal in the output telling you which you have received.
A dictionary is more limited and more honest. It gives you a word rather than a sentence, and when it has no entry it says so instead of inventing one.
Why we publish these at all
The Cheyenne dictionary will never recover its development cost, and neither will Ojibwe. They exist because the alternative for someone who needs them is very little — and because the communities using them have no larger dataset to fall back on.
It also means corrections matter more here than anywhere else in our catalogue. For a widely-supported language we could sanity-check against other tools. For these there is nothing to check against, so a message from a speaker pointing out an error is the best quality signal available, and it genuinely improves the app.
Dictionaries where automated tools are thin
45 offline dictionary pairs, including Khmer, Lao, Mizo, Hakha Chin, Hmong, Cheyenne and Ojibwe. Free, and working with no connection.
Open the dictionary appFrequently asked questions
Why is machine translation worse for some languages?
Because quality depends on how much parallel text exists to learn from, and that varies enormously. French and Spanish have decades of translated documents — parliamentary records, treaties, subtitles. A language with tens of millions of speakers but little digitised translated text has far less for a system to learn from, regardless of its speaker count.
What does under-resourced mean?
It describes a language with little machine-readable material — few digitised texts, limited dictionaries, no large corpora. It is not the same as endangered. Mizo has several hundred thousand speakers and official status in an Indian state, and is still thinly represented in digital tools.
Is a dictionary better than a translation app?
They do different jobs. A translation app gives you one answer quickly, which is what you want when you need to communicate now. A dictionary shows the range of senses with examples, which is what you want when you are learning. For under-resourced languages a dictionary is also often the more reliable of the two, because its entries were compiled by people.