The phrase-level fallback is a sensible contract: an uncertain token should not leave a half-declined dedication. I would preserve separators as well as casing when reconstructing accepted forms. Joining every token with a space can change a hyphenated name, an apostrophe, or an initial even when each word was inflected correctly.
Token spans can carry the original separators, with only the word spans replaced. A small boundary guard is also worth adding to the illustrative Russian suffix rule: the final-a branch reads w[-2], so a one-character input needs handling before that lookup. These cases belong beside the linguistic trap list because they test input preservation rather than morphology.
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
The phrase-level fallback is a sensible contract: an uncertain token should not leave a half-declined dedication. I would preserve separators as well as casing when reconstructing accepted forms. Joining every token with a space can change a hyphenated name, an apostrophe, or an initial even when each word was inflected correctly.
Token spans can carry the original separators, with only the word spans replaced. A small boundary guard is also worth adding to the illustrative Russian suffix rule: the final-a branch reads w[-2], so a one-character input needs handling before that lookup. These cases belong beside the linguistic trap list because they test input preservation rather than morphology.