---
title: "Discussion on \"Paged Attention in AI Inference\" | Hashnode"
description: "Discussion on \"Paged Attention in AI Inference\". Learn how paged attention cuts KV cache waste. Build a mini version in PyTorch and benchmark memory use against a contiguous cache."
url: "https://hashnode.com/posts/paged-attention-in-ai-inference/6aeed0a74f4b146adf854c72"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Paged Attention in AI Inference](https://ai-inference-pratyush.hashnode.dev/paged-attention-in-ai-inference?utm_source=hashnode&utm_medium=feed)

Learn how paged attention cuts KV cache waste. Build a mini version in PyTorch and benchmark memory use against a contiguous cache.

4d ago · 9 min read · ai-inference-pratyush.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#inference](https://hashnode.com/tag/inference)
-   [#ai-inference](https://hashnode.com/tag/ai-inference)
-   [#kv-cache](https://hashnode.com/tag/kv-cache)
-   [#kvcache](https://hashnode.com/tag/kvcache)
-   [#pagedattention](https://hashnode.com/tag/pagedattention)
-   [#llm](https://hashnode.com/tag/llm)
-   [#transformers](https://hashnode.com/tag/transformers)

## Written by

[Pratyush Ranjan](https://hashnode.com/@pratyushranjan02)

@pratyushranjan02 · Bengaluru, India

  
[View profile](https://hashnode.com/@pratyushranjan02)

Sharing interesting engineering stuff

Software engineer. Eager to learn storage engines, distributed databases, AI inference

## Responses 1

Markdown supported · ⌘↵ to post

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[2d ago](https://hashnode.com/posts/paged-attention-in-ai-inference/6aeed0a74f4b146adf854c72/comment/6af155a1922148e5e7498392)

The log-normal simulation would be useful with a block-size sweep as well. A typical 40-token sequence occupies 48 slots with 16-token blocks, but 64 slots with 64-token blocks, so the tail waste depends on the page size even when allocation is perfectly on demand.

I would report that unused tail per sequence alongside the aggregate cache utilization, and include block-table overhead when holding the total memory budget fixed. The three length distributions you already use would then show where smaller blocks save payload space and where their extra mapping metadata starts to matter.