---
title: "Comment by Alrebro on \"Why AI replies stream word-by-word\" | Hashnode"
description: "Great explainer — the distinction between the model being sequential and the transport being chunked is one most people conflate. The part about partial markdow"
url: "https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315/comment/6ac351f380c519aa00488282"
---

[Discussion](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315)

Comment [Why AI replies stream word-by-word](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315)

[Alrebro](https://hashnode.com/@alrebro)

[Oct 5](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315/comment/6ac351f380c519aa00488282)

Great explainer — the distinction between the model being sequential and the transport being chunked is one most people conflate. The part about partial markdown rendering resonated: tracking which structures are still open instead of naively re-parsing after every chunk is exactly the kind of detail that separates polished chat UIs from broken ones. One related wrinkle worth mentioning: token healing at stream boundaries. When a chunk boundary splits a multi-byte character or a token mid-word, naive concatenation can produce subtly wrong text even after all bytes arrive. Same class of problem as your UTF-8 example, just one layer up. Really enjoyed the Eloquent mention too — nice to see research on resumable streams getting attention.

[Nagappan S](https://hashnode.com/@imkarthik03)

[Oct 5](https://hashnode.com/posts/why-ai-replies-stream-word-by-word/6ac34ccf409651402523c315/comment/6ac351f380c519aa00488282#reply-6ac35c71b99a694287ac20f9)

Good addition. One nuance: token healing usually refers to fixing awkward token splits at the prompt/completion boundary, not stream chunking. For streaming, if you buffer raw bytes and only decode complete UTF-8 sequences, the final text matches the non-streamed output exactly.

The nearby problem that does bite is incremental detokenization. With some tokenizers, decoding token by token can differ from decoding the full ID sequence (leading-space handling, byte-fallback tokens), so UIs should decode over the accumulated IDs and emit only the stable prefix. Have you run into that one in practice?
