--- title: "#llm-inference articles | Hashnode" description: "Explore 15 articles tagged #llm-inference on Hashnode. Tutorials, guides, and how-tos from the community." url: "https://hashnode.com/tag/llm-inference" --- Tag # #llm-inference 15 posts · 0 followers [Akshay Suryawanshi](https://hashnode.com/@akshaysuryawanshi) · [5h ago](https://hashnode.com/posts/resizing-vllm-s-kv-cache-without-restarting-the-server/6af533f9230ef56d408e4992) ### [Resizing vLLM’s KV Cache Without Restarting the Server](https://akshaysuryawanshi.hashnode.dev/resizing-vllm-s-kv-cache-without-restarting-the-server?utm_source=hashnode&utm_medium=feed) How I built a vLLM plugin to resize KV cache without restarting, using drain and preserve strategies, CUDA VMM, and Qwen3-32B tests on an H100. 13 min read · [akshaysuryawanshi.hashnode.dev](https://akshaysuryawanshi.hashnode.dev/resizing-vllm-s-kv-cache-without-restarting-the-server?utm_source=hashnode&utm_medium=feed) [Prem Choudhary](https://hashnode.com/@prem14choudhary) · [Oct 28](https://hashnode.com/posts/safe-secrets-and-smart-scaling-in-eks-with-karpenter/6ae1fe675f98433cb3b3b0c8) ### [Safe Secrets and Smart Scaling in EKS with Karpenter](https://prem14choudhary.hashnode.dev/safe-secrets-and-smart-scaling-in-eks-with-karpenter?utm_source=hashnode&utm_medium=feed) Safe Secrets and Smart Scaling in EKS with Karpenter 9 min read · [prem14choudhary.hashnode.dev](https://prem14choudhary.hashnode.dev/safe-secrets-and-smart-scaling-in-eks-with-karpenter?utm_source=hashnode&utm_medium=feed) [Prem Choudhary](https://hashnode.com/@prem14choudhary) · [Oct 28](https://hashnode.com/posts/observability-stack-on-eks-via-terraform/6ae202e0a36b0d92103cb032) ### [Observability Stack on EKS via TERRAFORM....!!](https://prem14choudhary.hashnode.dev/observability-stack-on-eks-via-terraform?utm_source=hashnode&utm_medium=feed) Observability Stack on EKS via TERRAFORM 11 min read · [prem14choudhary.hashnode.dev](https://prem14choudhary.hashnode.dev/observability-stack-on-eks-via-terraform?utm_source=hashnode&utm_medium=feed) [Akshay Suryawanshi](https://hashnode.com/@akshaysuryawanshi) · [Oct 16](https://hashnode.com/posts/from-replicas-to-fp8-my-first-vllm-performance-investigation/6ad2ae0da305ffe6691825e0) ### [From Replicas to FP8: My First vLLM Performance Investigation](https://akshaysuryawanshi.hashnode.dev/from-replicas-to-fp8-my-first-vllm-performance-investigation?utm_source=hashnode&utm_medium=feed) Testing vLLM replicas, NVIDIA MPS, and FP8 KV cache and attention on one laptop GPU, measuring throughput, latency, goodput, and answer quality. 10 min read · [akshaysuryawanshi.hashnode.dev](https://akshaysuryawanshi.hashnode.dev/from-replicas-to-fp8-my-first-vllm-performance-investigation?utm_source=hashnode&utm_medium=feed) [schieh710](https://hashnode.com/@swchieh) · [Sep 25](https://hashnode.com/posts/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off/6ab6065e7b4232acdc17b980) ### [I Built a 9× Faster LLM Memory Allocator in C++ — and Worked Out When It Pays Off](https://seng-wei-chieh.hashnode.dev/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off?utm_source=hashnode&utm_medium=feed) A paged KV-cache allocator for Python, with every result checked against predictions I wrote before coding 18 min read · [seng-wei-chieh.hashnode.dev](https://seng-wei-chieh.hashnode.dev/i-built-a-9-faster-llm-memory-allocator-in-c-and-worked-out-when-it-pays-off?utm_source=hashnode&utm_medium=feed) [DaShaun](https://hashnode.com/@dashaun) · [Sep 15](https://hashnode.com/posts/two-dgx-sparks-one-model-deepseek-v4-flash-with-a-1m-token-context/6aa97ecc9e408ed2c912ffef) ### [Two DGX Sparks, One Model: DeepSeek-V4-Flash with a 1M-Token Context](https://dashaun.hashnode.dev/two-dgx-sparks-one-model-deepseek-v4-flash-with-a-1m-token-context?utm_source=hashnode&utm_medium=feed) Two NVIDIA DGX Sparks, one DeepSeek-V4-Flash model, and a full 1M-token context. The story behind the repo, the 30 MiB near-miss, and what I learned. 10 min read · [dashaun.hashnode.dev](https://dashaun.hashnode.dev/two-dgx-sparks-one-model-deepseek-v4-flash-with-a-1m-token-context?utm_source=hashnode&utm_medium=feed) [Manu Shukla](https://hashnode.com/@mr-manushukla) · [Aug 21](https://hashnode.com/posts/groq-retired-llama-33-70b-versatile-on-16-august-2026-and-points-production-teams-at-a-preview-model/6a881e9bbd7dad72a66c8c8a) ### [Groq retired llama-3.3-70b-versatile on 16 August 2026 and points production teams at a Preview model](https://ecorpit.hashnode.dev/groq-retired-llama-33-70b-versatile-on-16-august-2026-and-points-production-teams-at-a-preview-model?utm_source=hashnode&utm_medium=feed) Groq retired llama-3.3-70b-versatile on 16 August 2026 and points production teams at a Preview model Summary. On 16 August 2026 Groq shut down llama-3.1-8b-instant and llama-3.3-70b-versatile for fre 13 min read · [ecorpit.hashnode.dev](https://ecorpit.hashnode.dev/groq-retired-llama-33-70b-versatile-on-16-august-2026-and-points-production-teams-at-a-preview-model?utm_source=hashnode&utm_medium=feed) [Darsh Shah](https://hashnode.com/@darshs) · [Aug 17](https://hashnode.com/posts/how-to-scale-llm-inference-for-ai-agents-using-vllm/6a8373e0eb96152ac540effb) ### [How to Scale LLM Inference for AI Agents Using vLLM](https://freecodecamp.org/news/how-to-scale-llm-inference-for-ai-agents-using-vllm?utm_source=hashnode&utm_medium=feed) In this tutorial, I’ll show you how to scale LLM inference for AI agents using vLLM. I'll help you build an intuition for how LLM inference works, explore why agent workloads create GPU scheduling and 15 min read · [freecodecamp.org](https://freecodecamp.org/news/how-to-scale-llm-inference-for-ai-agents-using-vllm?utm_source=hashnode&utm_medium=feed) [Manu Shukla](https://hashnode.com/@mr-manushukla) · [Aug 2](https://hashnode.com/posts/vllm-speculative-decoding-in-2026-p-eagle-vs-dflash-vs-dspark-for-faster-cheaper-llm-serving/6a6fceede45372578ef537b2) ### [vLLM Speculative Decoding in 2026: P-EAGLE vs DFlash vs DSpark for Faster, Cheaper LLM Serving](https://ecorpit.hashnode.dev/vllm-speculative-decoding-in-2026-p-eagle-vs-dflash-vs-dspark-for-faster-cheaper-llm-serving?utm_source=hashnode&utm_medium=feed) vLLM speculative decoding in 2026: P-EAGLE vs DFlash vs DSpark for faster, cheaper LLM serving Summary. On July 28, 2026, the vLLM project and its Speculators library shipped open-source support for t 15 min read · [ecorpit.hashnode.dev](https://ecorpit.hashnode.dev/vllm-speculative-decoding-in-2026-p-eagle-vs-dflash-vs-dspark-for-faster-cheaper-llm-serving?utm_source=hashnode&utm_medium=feed) [Manu Shukla](https://hashnode.com/@mr-manushukla) · [Jul 19](https://hashnode.com/posts/nvidia-b200-vs-h100-in-2026-why-the-pricier-gpu-is-cheaper-per-token/6a5c5ab437150c5e13f80116) ### [NVIDIA B200 vs H100 in 2026: why the pricier GPU is cheaper per token](https://ecorpit.hashnode.dev/nvidia-b200-vs-h100-in-2026-why-the-pricier-gpu-is-cheaper-per-token?utm_source=hashnode&utm_medium=feed) NVIDIA B200 vs H100 in 2026: why the pricier GPU is cheaper per token Summary. In MLPerf Inference v6.0 (April 2026), a single NVIDIA B200 pushed about 17,500 tokens per second on Llama 2 70B in offli 12 min read · [ecorpit.hashnode.dev](https://ecorpit.hashnode.dev/nvidia-b200-vs-h100-in-2026-why-the-pricier-gpu-is-cheaper-per-token?utm_source=hashnode&utm_medium=feed)