VViswaretas Kotra·4d agoMeasure GPU startup to the first usable resultA GPU worker can be reachable while the useful result is still seconds away. If a benchmark stops at “ready,” it needs to say what became ready. Here is a deliberately fictional example from our runna3 min read·nodus-compute.hashnode.dev
KK. L. Aadithya·4d agoHow I Built an LLM Inference Engine From Scratch in CUDA (235 tokens/s on a 6 GB Laptop GPU)I built tinyinfer, an open-source LLM inference engine in C++/CUDA with no PyTorch or cuBLAS. It runs TinyLlama at 235 tokens/s on a 6 GB RTX 3050 laptop GPU.11 min read·aadithya.hashnode.dev
VViswaretas Kotra·4d agoA GPU partition changes the baselineA runnable fictional example shows why partition profiles, sibling load and timing uncertainty belong in a kernel benchmark baseline.3 min read·nodus-compute.hashnode.dev
VViswaretas Kotra·5d agoThree GPU timing numbers can all describe different boundariesA runnable synthetic worksheet separates GPU timing boundaries, overlap and incompatible clocks. Eight cases retained; no hardware measurements.4 min read·nodus-compute.hashnode.dev
AAkshay Suryawanshi·6d agoResizing vLLM’s KV Cache Without Restarting the ServerHow I built a vLLM plugin to resize KV cache without restarting, using drain and preserve strategies, CUDA VMM, and Qwen3-32B tests on an H100.13 min read·akshaysuryawanshi.hashnode.dev
AAnshu Gondi·Nov 4Why vLLM, SGLang, and FlashAttention Are Just 1970s Embedded Systems AlgorithmsDemystifying GPU inference primitives by tracing FlashAttention, PagedAttention, and KV-cache reuse back to classic operating system memory paging and hardware scratchpads.3 min read·anshugondi.hashnode.dev
AAleksei Romanov·Nov 3Teaching a 27B Model to Write Triton Kernels: SFT + RL GymIn Part 1, we explored how GPU kernels work under the hood — why operator fusion and memory tiling matter, and how benchmarks like KernelBench evaluate whether an LLM can write kernels that actually b27 min read·gfactor.hashnode.dev
AAleksei Romanov·Nov 3GPU Kernels, Explained: CUDA, Triton, cuBLAS, torch.compile and KernelBenchWhen you run a neural network in PyTorch, Python is only the coordinator. Behind every tensor operation — from a simple activation to a multi-head attention block — sits one or more GPU kernels: speci28 min read·gfactor.hashnode.dev
AAnshu Gondi·Oct 27Here is the complete, publication-ready Hashnode article formatted cleanly in Markdown. You can paste this directly into Hashnode's editor. Stripping Here is the complete, publication-ready Hashnode article formatted cleanly in Markdown. You can paste this directly into Hashnode's editor. Stripping the Python Tax: How We Ported vLLM Concepts direct3 min read·anshugondi.hashnode.dev
AAman Karki·Oct 26My GPU fix was 13% faster on one card and 81% slower on anotherA vLLM int8 GEMM change, a reviewer with a smaller Ada GPU, and the one idea that explained it: waves7 min read·amankarki.hashnode.dev