Anton R Gordon on Why Modern LLMs Think in Context, Not Sequences: The Evolution Beyond Self-Attention

When the Transformer architecture was introduced in the landmark 2017 paper Attention Is All You Need, it fundamentally changed how machines process language. Instead of reading text one token at a ti