AAnand·4d agoSomeone Else's Words in Your Answer: A Design Review of Cloudflare's GLM-5.3 KV Cache ExposureHow a refused prompt crossed between two machines and came back with other customers' words, what Cloudflare and SGLang changed, and what's still open.27 min read·anandms101.hashnode.dev2AS
PPratyush Ranjan·4d agoPaged Attention in AI InferenceLearn how paged attention cuts KV cache waste. Build a mini version in PyTorch and benchmark memory use against a contiguous cache.9 min read·ai-inference-pratyush.hashnode.dev1A
PPratyush Ranjan·Oct 26KV Cache Optimisation in AI InferenceLearn how the KV cache speeds up LLM inference. We build a small GPT in PyTorch and benchmark cached vs uncached generation: a 4.5x speedup.8 min read·ai-inference-pratyush.hashnode.dev11A
TTidiane Stano·Oct 24Understanding the "Memory" of Large Language Models: What Exactly Is KV Cache?Learn how KV cache speeds up LLM decoding, why it consumes GPU memory, and how GQA, quantization, and PagedAttention help.11 min read·4sapi.hashnode.dev
SSundarapandy·Oct 23How KV Cache Reduces Redundant Computation During LLM Token GenerationExplore how KV Cache reduces redundant computation in LLM inference, speeds up token generation, and manages memory in production systems efficiently.3 min read·ai-tech-insights.hashnode.dev1
Sschieh710·Sep 25I Built a 9× Faster LLM Memory Allocator in C++ — and Worked Out When It Pays OffA paged KV-cache allocator for Python, with every result checked against predictions I wrote before coding18 min read·seng-wei-chieh.hashnode.dev
RRahul Sai Indeevar V·Sep 7Advanced Transformers: Positional Encodings (RoPE) & Efficient Attention Mechanisms (KV Caching, MQA, GQA, MLA)In modern Large Language Model (LLM) architectures, two foundational engineering bottlenecks dictate model scalability and inference latency: Representing Order: How self-attention mechanisms—which a15 min read·rahul-ai.hashnode.dev13A
DDarsh Shah·Aug 17How to Scale LLM Inference for AI Agents Using vLLMIn this tutorial, I’ll show you how to scale LLM inference for AI agents using vLLM. I'll help you build an intuition for how LLM inference works, explore why agent workloads create GPU scheduling and15 min read·freecodecamp.org1
GGraham Andanje·Aug 6Cache EvictionCache eviction for optimizing KV Cache memory size9 min read·fata-fanaka.hashnode.dev
NNguyễn Hữu Đức·Jul 11Building an LLM System from Scratch in Pure Python & NumPy: Architecture, Invariants, and Clean Code, Open-Source 100% Inference Engine and Thinking engine!Exploring the design invariants, ternary quantization, and self-healing cognitive pipeline of Draco AI, a hardware-agnostic LLM built from scratch.3 min read·draco-ai.hashnode.dev
