AAkshay Suryawanshi·4h agoResizing vLLM’s KV Cache Without Restarting the ServerHow I built a vLLM plugin to resize KV cache without restarting, using drain and preserve strategies, CUDA VMM, and Qwen3-32B tests on an H100.13 min read·akshaysuryawanshi.hashnode.dev
Rreshmi aravind·19h agoFrom GPU PIDs to Model IDs: Which vLLM Model Is Using My GPU?When running LLM inference workloads, tools like nvidia-smi and nvtop help identify GPU utilization, memory consumption, and the processes using those resources. But they don't directly answer another11 min read·practical-ml-systems.hashnode.dev
SShubham Tatvamasi·2d agovLLM Deep Dive: How PagedAttention, Continuous Batching and the V1 Engine Serve LLMs in ProductionEvery layer above it routes, scales and governs requests. vLLM is the layer that turns GPU memory into tokens.52 min read·blog.shubhamtatvamasi.com63CKS
4404 Founders·Nov 5vLLM CORS Vulnerability: The API Key Never Covered /invocationsvLLM before 0.32.0 let any web page read model output via the browser, even with --api-key set. Checks and fixes for CVE-2026-115917.15 min read·404-founders.com
AAndolasoft Inc·Nov 2Air-Gapped AI Analytics: Apache Superset with Local LLMs (Ollama and vLLM)Somebody senior has seen a demo. A business user types “which branches had the worst collection efficiency last quarter” and a chart appears. Now they want it in your bank, your ministry or your hospi2 min read·andolasoft.hashnode.dev
AAman Karki·Oct 26My GPU fix was 13% faster on one card and 81% slower on anotherA vLLM int8 GEMM change, a reviewer with a smaller Ada GPU, and the one idea that explained it: waves7 min read·amankarki.hashnode.dev
RRuncrate·Oct 26vLLM on three GPUs: Qwen2.5-7B tokens per second and cost per 1M tokens on RTX A6000, L40S and H100Same model, same benchmark, three rented GPUs: $0.12 to $0.22 per 1M output tokens when the batch is full, $4.86 to $9.47 when it is not.6 min read·runcrate.hashnode.dev
TTheaicommit·Oct 23Why Split Prefill and Decode GPUs Lose Time in the Hand-Off, Not the NetworkMoving the KV cache takes 10 ms, but first-token time hit 1.7 s. How vLLM's pull hand-off stalls the decode loop, and how to size decode slots and fix it.10 min read·theaicommit.hashnode.dev
AAman Karki·Oct 17Blockwise FP8 on Ada GPUs, part 2: my kernel was correct and 4x too slowHow I found register spills in the main loop, fixed them, and got a 1.42x end-to-end speedup on an L48 min read·amankarki.hashnode.dev
AAkshay Suryawanshi·Oct 16From Replicas to FP8: My First vLLM Performance InvestigationTesting vLLM replicas, NVIDIA MPS, and FP8 KV cache and attention on one laptop GPU, measuring throughput, latency, goodput, and answer quality.10 min read·akshaysuryawanshi.hashnode.dev
