AI Can't Count the R's in 'Strawberry.' One Fact Explains Why — and a Dozen Other Bugs.

This is an excellent addition. The “token cost” of multilingual AI is easy to overlook because we tend to benchmark everything against English.

And I especially like your point about misdiagnosing the problem. When retrieval quality drops, it’s tempting to blame the embedding model or search layer, when tokenization may already be introducing fragmentation upstream.

So the strawberry bug isn’t just a curiosity—it’s a reminder that seemingly small tokenizer decisions can have real consequences for cost, context, and multilingual performance. Thanks for adding this perspective!

I should qualify the embedding part of my earlier comment: heavier fragmentation does not by itself prove worse retrieval. The encoder's training and preprocessing also matter, and the effect needs to be measured rather than inferred from token count alone.

A useful comparison would log whether matched-language passages were truncated under the encoder's actual limit, then evaluate retrieval with and without that truncation. That separates a concrete loss of source text from a broader language-quality issue. The cost and context-budget consequences remain directly measurable, but the retrieval explanation needs its own evidence.