How KV Cache and Paged Attention Unlock Faster, Cheaper LLM Inference

TL;DR: LLM inference bottlenecks are usually memory, not compute. KV cache eliminates redundant attention computation. Paged attention eliminates memory fragmentation — cutting waste from ~60–80% down