---
title: "Discussion on \"An Attention Window Is Not a Checkpoint: Inside vLLM's 128-Token Replay\" | Hashnode"
description: "Discussion on \"An Attention Window Is Not a Checkpoint: Inside vLLM's 128-Token Replay\". A coding agent returns with another test result. Following that request reveals why vLLM saves less state between turns, then rebuilds a recent window without reproducing it exactly."
url: "https://hashnode.com/posts/an-attention-window-is-not-a-checkpoint-inside-vllm-s-128-token-replay/6acb2337061765bf8144b005"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [An Attention Window Is Not a Checkpoint: Inside vLLM's 128-Token Replay](https://anandms101.hashnode.dev/an-attention-window-is-not-a-checkpoint-inside-vllm-s-128-token-replay?utm_source=hashnode&utm_medium=feed)

A coding agent returns with another test result. Following that request reveals why vLLM saves less state between turns, then rebuilds a recent window without reproducing it exactly.

Oct 11 · 10 min read · anandms101.hashnode.dev  
Read the article

-   [#system-design](https://hashnode.com/tag/system-design)
-   [#artificial-intelligence](https://hashnode.com/tag/artificial-intelligence)
-   [#llm](https://hashnode.com/tag/llm)
-   [#vllm](https://hashnode.com/tag/vllm)
-   [#deepseek](https://hashnode.com/tag/deepseek)

## Written by

[Anand](https://hashnode.com/@anandms)

@anandms · Redmond, WA

  
[View profile](https://hashnode.com/@anandms)

Software Engineer II @ Microsoft

I write about AI/ML and DSA topics

## Responses 2

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 12](https://hashnode.com/posts/an-attention-window-is-not-a-checkpoint-inside-vllm-s-128-token-replay/6acb2337061765bf8144b005/comment/6acd2ddcdb7691c9e42932ef)

Opening on the coding-agent task that keeps failing and then going into the 128-token replay is a good way in, because I think most people treat the attention window like durable state when it is not. The distinction from a checkpoint is worth hammering: nothing about the window survives the way a real checkpoint would, so "it remembered a second ago" is doing a lot of unearned work. Did the retry-test example actually trace back to the replay boundary, or was that mostly the hook?

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 11](https://hashnode.com/posts/an-attention-window-is-not-a-checkpoint-inside-vllm-s-128-token-replay/6acb2337061765bf8144b005/comment/6acb5b01b19fbe5ad15d693c)

The four, one, zero illustration separates dependency completeness from answer accuracy carefully. For an empirical follow-up, I would compare a cold computation and an eligible prefix-cache hit using the same token sequence, then inspect next-token distributions before comparing whole generated answers.

A deterministic decode can remain identical despite a measurable distribution shift, while sampling can diverge for reasons unrelated to replay. Recording cache eligibility and the reused-prefix length would keep those conditions distinct. That would test the approximation's practical effect without turning the local ancestry diagram into an unmeasured claim about an agent forgetting its instruction.
