Your Local MoE Model Is Slow: A Diagnostic Checklist
Mixture-of-experts models are the reason a 30B can decode like a 3B on a desktop. They are also the reason people post "why is my local model crawling" threads, because MoE has a handful of failure mo
joshgreen.hashnode.dev7 min read