Cache Eviction

The key-value cache is an inference optimization technique that eliminates redundant recomputation of past token representations during autoregressive generation. The KV cache memory footprint scales