Paged Attention in AI InferenceLearn how paged attention cuts KV cache waste. Build a mini version in PyTorch and benchmark memory use against a contiguous cache.9 min read·ai-inference-pratyush.hashnode.dev1ANov 7
KV Cache Optimisation in AI InferenceLearn how the KV cache speeds up LLM inference. We build a small GPT in PyTorch and benchmark cached vs uncached generation: a 4.5x speedup.8 min read·ai-inference-pratyush.hashnode.dev11AOct 26