JGJosh Greeninjoshgreen.hashnode.dev·3d ago · 7 min readYour Local MoE Model Is Slow: A Diagnostic ChecklistMixture-of-experts models are the reason a 30B can decode like a 3B on a desktop. They are also the reason people post "why is my local model crawling" threads, because MoE has a handful of failure mo00
JGJosh Greeninjoshgreen.hashnode.dev·6d ago · 8 min readGLM 4.7 Flash on Two RTX 3090s: The Harness, Every Config, and a Decision TableI keep two RTX 3090s in a desktop, not a server. No NVLink, a consumer motherboard, the kind of build a lot of people actually have sitting under a desk. The question I wanted answered was narrow and 00
JGJosh Greeninjoshgreen.hashnode.dev·Jun 6 · 8 min readWhy DDR5 Bandwidth Kills Dual-LLM Inference on APUs (Benchmarks Inside)Did you know that a 35-billion-parameter model can generate tokens at the same compute cost as a 4B model? That single fact made me abandon a multi-model agent architecture I'd spent a weekend buildin10