--- title: "#kv-cache articles | Hashnode" description: "Explore 16 articles tagged #kv-cache on Hashnode. Tutorials, guides, and how-tos from the community." url: "https://hashnode.com/tag/kv-cache" --- Tag # #kv-cache 16 posts · 0 followers [Anand](https://hashnode.com/@anandms) · [4d ago](https://hashnode.com/posts/someone-else-s-words-in-your-answer-a-design-review-of-cloudflare-s-glm-5-3-kv-cache-exposure/6aee9c260df03d4b598b1c33) ### [Someone Else's Words in Your Answer: A Design Review of Cloudflare's GLM-5.3 KV Cache Exposure](https://anandms101.hashnode.dev/someone-else-s-words-in-your-answer-a-design-review-of-cloudflare-s-glm-5-3-kv-cache-exposure?utm_source=hashnode&utm_medium=feed) How a refused prompt crossed between two machines and came back with other customers' words, what Cloudflare and SGLang changed, and what's still open. 27 min read · [anandms101.hashnode.dev](https://anandms101.hashnode.dev/someone-else-s-words-in-your-answer-a-design-review-of-cloudflare-s-glm-5-3-kv-cache-exposure?utm_source=hashnode&utm_medium=feed) [2](https://hashnode.com/posts/someone-else-s-words-in-your-answer-a-design-review-of-cloudflare-s-glm-5-3-kv-cache-exposure/6aee9c260df03d4b598b1c33) [Pratyush Ranjan](https://hashnode.com/@pratyushranjan02) · [4d ago](https://hashnode.com/posts/paged-attention-in-ai-inference/6aeed0a74f4b146adf854c72) ### [Paged Attention in AI Inference](https://ai-inference-pratyush.hashnode.dev/paged-attention-in-ai-inference?utm_source=hashnode&utm_medium=feed) Learn how paged attention cuts KV cache waste. Build a mini version in PyTorch and benchmark memory use against a contiguous cache. 9 min read · [ai-inference-pratyush.hashnode.dev](https://ai-inference-pratyush.hashnode.dev/paged-attention-in-ai-inference?utm_source=hashnode&utm_medium=feed) [1](https://hashnode.com/posts/paged-attention-in-ai-inference/6aeed0a74f4b146adf854c72) [Pratyush Ranjan](https://hashnode.com/@pratyushranjan02) · [Oct 26](https://hashnode.com/posts/kv-cache-optimisation-in-ai-inference/6adf5f5e5bb6fa9a24a51c06) ### [KV Cache Optimisation in AI Inference](https://ai-inference-pratyush.hashnode.dev/kv-cache-optimisation-in-ai-inference?utm_source=hashnode&utm_medium=feed) Learn how the KV cache speeds up LLM inference. We build a small GPT in PyTorch and benchmark cached vs uncached generation: a 4.5x speedup. 8 min read · [ai-inference-pratyush.hashnode.dev](https://ai-inference-pratyush.hashnode.dev/kv-cache-optimisation-in-ai-inference?utm_source=hashnode&utm_medium=feed) [1](https://hashnode.com/posts/kv-cache-optimisation-in-ai-inference/6adf5f5e5bb6fa9a24a51c06) [Tidiane Stano](https://hashnode.com/@4sapi) · [Oct 24](https://hashnode.com/posts/understanding-the-memory-of-large-language-models-what-exactly-is-kv-cache/6adc693ce981d0df3a444052) ### [Understanding the "Memory" of Large Language Models: What Exactly Is KV Cache?](https://4sapi.hashnode.dev/understanding-the-memory-of-large-language-models-what-exactly-is-kv-cache?utm_source=hashnode&utm_medium=feed) Learn how KV cache speeds up LLM decoding, why it consumes GPU memory, and how GQA, quantization, and PagedAttention help. 11 min read · [4sapi.hashnode.dev](https://4sapi.hashnode.dev/understanding-the-memory-of-large-language-models-what-exactly-is-kv-cache?utm_source=hashnode&utm_medium=feed) [Sundarapandy](https://hashnode.com/@sundarapandy) · [Oct 23](https://hashnode.com/posts/how-kv-cache-reduces-redundant-computation-during-llm-token-generation/6adb4ebb00048ff8f6e4ec4e) ### [How KV Cache Reduces Redundant Computation During LLM Token Generation](https://ai-tech-insights.hashnode.dev/how-kv-cache-reduces-redundant-computation-during-llm-token-generation?utm_source=hashnode&utm_medium=feed) Explore how KV Cache reduces redundant computation in LLM inference, speeds up token generation, and manages memory in production systems efficiently. 3 min read · [ai-tech-insights.hashnode.dev](https://ai-tech-insights.hashnode.dev/how-kv-cache-reduces-redundant-computation-during-llm-token-generation?utm_source=hashnode&utm_medium=feed) [schieh710](https://hashnode.com/@swchieh) · [Sep 25](https://hashnode.com/posts/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off/6ab6065e7b4232acdc17b980) ### [I Built a 9× Faster LLM Memory Allocator in C++ — and Worked Out When It Pays Off](https://seng-wei-chieh.hashnode.dev/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off?utm_source=hashnode&utm_medium=feed) A paged KV-cache allocator for Python, with every result checked against predictions I wrote before coding 18 min read · [seng-wei-chieh.hashnode.dev](https://seng-wei-chieh.hashnode.dev/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off?utm_source=hashnode&utm_medium=feed) [Rahul Sai Indeevar V](https://hashnode.com/@rahul-ai) · [Sep 7](https://hashnode.com/posts/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla/6a9f04585a20161fc07430bd) ### [Advanced Transformers: Positional Encodings (RoPE) & Efficient Attention Mechanisms (KV Caching, MQA, GQA, MLA)](https://rahul-ai.hashnode.dev/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla?utm_source=hashnode&utm_medium=feed) In modern Large Language Model (LLM) architectures, two foundational engineering bottlenecks dictate model scalability and inference latency: Representing Order: How self-attention mechanisms—which a 15 min read · [rahul-ai.hashnode.dev](https://rahul-ai.hashnode.dev/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla?utm_source=hashnode&utm_medium=feed) [3](https://hashnode.com/posts/advanced-transformers-positional-encodings-rope-efficient-attention-mechanisms-kv-caching-mqa-gqa-mla/6a9f04585a20161fc07430bd) [Darsh Shah](https://hashnode.com/@darshs) · [Aug 17](https://hashnode.com/posts/how-to-scale-llm-inference-for-ai-agents-using-vllm/6a8373e0eb96152ac540effb) ### [How to Scale LLM Inference for AI Agents Using vLLM](https://freecodecamp.org/news/how-to-scale-llm-inference-for-ai-agents-using-vllm?utm_source=hashnode&utm_medium=feed) In this tutorial, I’ll show you how to scale LLM inference for AI agents using vLLM. I'll help you build an intuition for how LLM inference works, explore why agent workloads create GPU scheduling and 15 min read · [freecodecamp.org](https://freecodecamp.org/news/how-to-scale-llm-inference-for-ai-agents-using-vllm?utm_source=hashnode&utm_medium=feed) [Graham Andanje](https://hashnode.com/@fata-fanaka) · [Aug 6](https://hashnode.com/posts/cache-eviction/6a7461385c97f3cfab7f6c44) ### [Cache Eviction](https://fata-fanaka.hashnode.dev/cache-eviction?utm_source=hashnode&utm_medium=feed) Cache eviction for optimizing KV Cache memory size 9 min read · [fata-fanaka.hashnode.dev](https://fata-fanaka.hashnode.dev/cache-eviction?utm_source=hashnode&utm_medium=feed) [Nguyễn Hữu Đức](https://hashnode.com/@ducnguyen-creator) · [Jul 11](https://hashnode.com/posts/draco-ai/6a51b42422570cdfac8f9efd) ### [Building an LLM System from Scratch in Pure Python & NumPy: Architecture, Invariants, and Clean Code, Open-Source 100% Inference Engine and Thinking engine!](https://draco-ai.hashnode.dev/draco-ai?utm_source=hashnode&utm_medium=feed) Exploring the design invariants, ternary quantization, and self-healing cognitive pipeline of Draco AI, a hardware-agnostic LLM built from scratch. 3 min read · [draco-ai.hashnode.dev](https://draco-ai.hashnode.dev/draco-ai?utm_source=hashnode&utm_medium=feed)