Large Language Models · entry 05/11 · 3 min read

Models in other languages

Quality, cost and safety all degrade away from English — the tokenizer tax, the resource cliff, and why testing in one language tells you almost nothing about the others.

Three penalties, stacked

A model that is excellent in English is not the same product elsewhere, and the gap has three separate causes that compound. Data: pretraining corpora are overwhelmingly English, so knowledge and fluency thin out with a language's share of the web. Tokenization: a vocabulary learned mostly on English fragments other scripts badly — the same sentence can cost two to five times more tokens in Hindi or Thai than in English, and ten times or worse in scripts like Amharic or Burmese, which means it costs more money, fills the context window faster, and generates more slowly. Alignment: post-training data is even more English-skewed than pretraining data, so refusals, formatting and tone are tuned in one language and inherited unevenly by the rest.

The failure changes shape

Performance decays faster than it looks as a language's share of the training data falls — and, more dangerously, the shape of the failure changes with it. High-resource languages sit close to English. Mid-resource ones degrade in ways that are easy to miss — grammatical output, confident content, quietly wrong facts. Genuinely low-resource languages produce fluent nonsense, because fluency is cheap to learn from a little data while knowledge is not. That failure shape is dangerous precisely because it looks like the success shape, and it is invisible to anyone testing in English.

What helps

Cross-lingual transfer is real: models answer in languages they were barely trained on by reusing what they learned in high-resource ones through a largely language-agnostic internal representation, which is why they work at all outside the top twenty. Beyond that, the practical levers are ordinary. Retrieve in the user's language, then measure whether answering natively or pivoting through English wins on your task — for low-resource languages the pivot often does, counter-intuitive as that reads. Fine-tune for target-language fluency and format when volume justifies it, but retrieve for the knowledge gap, and prefer models whose tokenizer covers your script. Regionally-built open models often beat comparably sized open models on their own languages, and sometimes beat frontier ones on culturally specific tasks — mostly through the data mix, with tokenizer coverage cutting the bill.

Failure mode

Certifying in English and shipping globally. The eval suite is English, the red-team is English, the golden set is English — and the product launches in fourteen markets where none of that evidence applies. Jailbreak resistance in particular transfers poorly: attacks that fail in English often succeed when translated into a low-resource one, because that is where the safety training was thinnest — and fine-tuning erodes refusal behavior further, so re-run those checks in the target language after any tuning. Build the eval in every language you actually ship to, or be honest that you have shipped an untested product to everyone else.