How Token Limits Affect AI Reasoning in Long Prompts

We are currently in a "context war." Model providers are racing to ship 100k, 200k, and even 1M token context windows. The marketing implies a linear relationship: if a model can ingest a whole book, it can reason about the whole book. In production,...