AAkshay Suryawanshi·4h agoResizing vLLM’s KV Cache Without Restarting the ServerHow I built a vLLM plugin to resize KV cache without restarting, using drain and preserve strategies, CUDA VMM, and Qwen3-32B tests on an H100.13 min read·akshaysuryawanshi.hashnode.dev
PPrem Choudhary·Oct 28Safe Secrets and Smart Scaling in EKS with KarpenterSafe Secrets and Smart Scaling in EKS with Karpenter9 min read·prem14choudhary.hashnode.dev
PPrem Choudhary·Oct 28Observability Stack on EKS via TERRAFORM....!!Observability Stack on EKS via TERRAFORM11 min read·prem14choudhary.hashnode.dev
AAkshay Suryawanshi·Oct 16From Replicas to FP8: My First vLLM Performance InvestigationTesting vLLM replicas, NVIDIA MPS, and FP8 KV cache and attention on one laptop GPU, measuring throughput, latency, goodput, and answer quality.10 min read·akshaysuryawanshi.hashnode.dev
Sschieh710·Sep 25I Built a 9× Faster LLM Memory Allocator in C++ — and Worked Out When It Pays OffA paged KV-cache allocator for Python, with every result checked against predictions I wrote before coding18 min read·seng-wei-chieh.hashnode.dev
DDaShaun·Sep 15Two DGX Sparks, One Model: DeepSeek-V4-Flash with a 1M-Token ContextTwo NVIDIA DGX Sparks, one DeepSeek-V4-Flash model, and a full 1M-token context. The story behind the repo, the 30 MiB near-miss, and what I learned.10 min read·dashaun.hashnode.dev
MManu Shukla·Aug 21Groq retired llama-3.3-70b-versatile on 16 August 2026 and points production teams at a Preview modelGroq retired llama-3.3-70b-versatile on 16 August 2026 and points production teams at a Preview model Summary. On 16 August 2026 Groq shut down llama-3.1-8b-instant and llama-3.3-70b-versatile for fre13 min read·ecorpit.hashnode.dev
DDarsh Shah·Aug 17How to Scale LLM Inference for AI Agents Using vLLMIn this tutorial, I’ll show you how to scale LLM inference for AI agents using vLLM. I'll help you build an intuition for how LLM inference works, explore why agent workloads create GPU scheduling and15 min read·freecodecamp.org1
MManu Shukla·Aug 2vLLM Speculative Decoding in 2026: P-EAGLE vs DFlash vs DSpark for Faster, Cheaper LLM ServingvLLM speculative decoding in 2026: P-EAGLE vs DFlash vs DSpark for faster, cheaper LLM serving Summary. On July 28, 2026, the vLLM project and its Speculators library shipped open-source support for t15 min read·ecorpit.hashnode.dev
MManu Shukla·Jul 19NVIDIA B200 vs H100 in 2026: why the pricier GPU is cheaper per tokenNVIDIA B200 vs H100 in 2026: why the pricier GPU is cheaper per token Summary. In MLPerf Inference v6.0 (April 2026), a single NVIDIA B200 pushed about 17,500 tokens per second on Llama 2 70B in offli12 min read·ecorpit.hashnode.dev
