Putting RoPE and the KV cache family in one article makes sense because they are the two things that decide what long context actually costs you. One framing worth adding for readers coming from the serving side: MQA and GQA are memory bandwidth optimisations more than compute ones. Decoding is bandwidth-bound - you re-read the whole cache every token - so shrinking the number of KV heads speeds up generation even when FLOPs barely move, which surprises people who benchmark on prefill and see nothing. The practical consequence of the cache section is that serving economics stop being about tokens per second and start being about how many contexts fit in memory at once. Long-running agent workloads keep large caches resident across many steps, so cost per resident context-hour predicts your bill better than cost per token does. Worth noting on RoPE too that extending context at inference is not free - the positions the model never trained on behave differently, which is why scaling tricks exist and why long-context benchmarks disagree with each other so much.