Beyond Injectivity: What the ‘Language Models are Injective’ Paper Reveals—and What It Overlooks

If you’ve been anywhere near AI Twitter or arXiv this week, you’ve likely encountered a striking claim: Transformer language models are injective. Different prompts almost surely lead to different hidden states. Not only that, but we can exactly reco...