This hits Indian languages even harder than Russian. Hindi, Tamil or Bengali in Devanagari and other scripts can easily take 3-5x the tokens of the same English sentence on older tokenizers, so a support bot for Indian users pays a hidden language tax on every turn and runs out of context sooner. Did the author test any Indic scripts, or code-mixed text like Hinglish where people type Hindi in Latin letters?
iin1005h1128