5d ago · 10 min read · Every production LLM deployment using speculative decoding is likely running a fixed speculation length of γ=4. That number comes from early benchmarks, it has been copy-pasted across blog posts and framework defaults, and almost nobody questions it....
Join discussion
May 6 · 10 min read · TL;DR — I was happily running Qwen3.6 on llama.cpp. Then I saw claims of 2× speed with vLLM + NVFP4 + DFlash. So I installed it, fought through crashes, and measured it myself. Verdict: it's real. 88–
Join discussionMay 3 · 5 min read · The Windows local LLM story just got interesting. Someone recently demonstrated Qwen3's 27B model running at 72 tokens per second on an RTX 3090 — natively on Windows. No WSL. No Docker. Just a portable vLLM launcher. If you've been running local mod...
Join discussion
Apr 28 · 11 min read · Mixture-of-Experts models have dominated the open-weight frontier in 2026. Llama 4 Scout (17B-16E), Llama 4 Maverick (17B-128E), DeepSeek V4-Pro (1.6T-49B active), and Qwen3.6-Plus all use sparse expert routing to scale parameters without proportiona...
Join discussion
Apr 22 · 11 min read · Serving a large language model in production is a solved problem — until your traffic doubles, your structured output pipeline slows to a crawl, or your cloud bill arrives. The choice of inference engine determines how many GPUs you actually need, ho...
Join discussion
Apr 17 · 16 min read · TLDR: Most teams should start with managed LLM APIs because they buy speed, reliability, model quality, and low operational burden. Move to self-hosted or open-weight models only when you have stable workloads, hard privacy or compliance constraints,...
Join discussion
