--- title: "#mixture-of-experts articles | Hashnode" description: "Explore 18 articles tagged #mixture-of-experts on Hashnode. Tutorials, guides, and how-tos from the community." url: "https://hashnode.com/tag/mixture-of-experts" --- Tag # #mixture-of-experts 18 posts · 0 followers [Shivam Kumar](https://hashnode.com/@shivaysinghrajput) · [Oct 7](https://hashnode.com/posts/i-built-a-188m-mixture-of-experts-llm-from-scratch-on-a-free-gpu/6ac5d08edecbf46bd69bcc1e) ### [I Built a 188M Mixture-of-Experts LLM From Scratch on a Free GPU](https://shivaysinghrajput.hashnode.dev/i-built-a-188m-mixture-of-experts-llm-from-scratch-on-a-free-gpu?utm_source=hashnode&utm_medium=feed) By Shivam Kumar, founder of VisionQuantech. This is the honest version — what's proven, what's measured, and what's still running. Why a tiny MoE? Most mixture-of-experts research happens at billion-p 6 min read · [shivaysinghrajput.hashnode.dev](https://shivaysinghrajput.hashnode.dev/i-built-a-188m-mixture-of-experts-llm-from-scratch-on-a-free-gpu?utm_source=hashnode&utm_medium=feed) [1](https://hashnode.com/posts/i-built-a-188m-mixture-of-experts-llm-from-scratch-on-a-free-gpu/6ac5d08edecbf46bd69bcc1e) [mamonu](https://hashnode.com/@mamonu) · [Sep 10](https://hashnode.com/posts/running-large-moe-llms-on-modest-hardware-with-freetoken/6aa2a6107f57524dc8baf606) ### [Running Large MoE LLMs on Modest Hardware with FreeToken](https://mamonu.hashnode.dev/running-large-moe-llms-on-modest-hardware-with-freetoken?utm_source=hashnode&utm_medium=feed) A new open-source inference engine designed to run MoE models on hardware that doesn't have enough VRAM to hold them 15 min read · [mamonu.hashnode.dev](https://mamonu.hashnode.dev/running-large-moe-llms-on-modest-hardware-with-freetoken?utm_source=hashnode&utm_medium=feed) [Learn with HJ](https://hashnode.com/@hardeepjethwani) · [Jul 11](https://hashnode.com/posts/mixture-of-experts-why-some-models-use-specialist-sub-brains/6a5262b66aa432b551ce7b7e) ### [Mixture of Experts: Why Some Models Use Specialist Sub-Brains](https://hardeepjethwani.hashnode.dev/mixture-of-experts-why-some-models-use-specialist-sub-brains?utm_source=hashnode&utm_medium=feed) Explain MoE architecture simply, benefits, routing, cost, and limitations. 6 min read · [hardeepjethwani.hashnode.dev](https://hardeepjethwani.hashnode.dev/mixture-of-experts-why-some-models-use-specialist-sub-brains?utm_source=hashnode&utm_medium=feed) [Jangwook Kim](https://hashnode.com/@effloow) · [May 8](https://hashnode.com/posts/kimi-k2-6-agentic-coding-developer-guide-2026/69fd9c69b6b3c7bfd17130f7) ### [Kimi K2.6: The Open 1T-Param Model for Agentic Coding](https://effloow.hashnode.dev/kimi-k2-6-agentic-coding-developer-guide-2026?utm_source=hashnode&utm_medium=feed) On April 20, 2026, Moonshot AI released Kimi K2.6 under a Modified MIT license — a 1-trillion-parameter open-weight model that sits within percentage points of Claude Opus 4.6 on SWE-Bench Verified while being fully accessible through public APIs and... 10 min read · [effloow.hashnode.dev](https://effloow.hashnode.dev/kimi-k2-6-agentic-coding-developer-guide-2026?utm_source=hashnode&utm_medium=feed) [Anup Karanjkar](https://hashnode.com/@anupkaranjkar) · [May 2](https://hashnode.com/posts/llama-4-scout-local-deployment-moe-complete-guide-2026/69f582a49b45dc4a0c3c31ff) ### [Llama 4 Scout: Run a GPT-4-Class AI Locally for Free in 2026](https://wowhow.hashnode.dev/llama-4-scout-local-deployment-moe-complete-guide-2026?utm_source=hashnode&utm_medium=feed) Meta’s Llama 4 Scout launched April 2026 with a 10M token context window. Here’s the hardware specs, vLLM setup, and benchmark breakdown for local deployment. 9 min read · [wowhow.hashnode.dev](https://wowhow.hashnode.dev/llama-4-scout-local-deployment-moe-complete-guide-2026?utm_source=hashnode&utm_medium=feed) [Anup Karanjkar](https://hashnode.com/@anupkaranjkar) · [May 2](https://hashnode.com/posts/deepseek-v4-1-trillion-parameters-what-it-means-for-ai/69f583397f5d81b9a4552533) ### [DeepSeek V4 is Coming: What 1 Trillion Parameters Means for AI](https://wowhow.hashnode.dev/deepseek-v4-1-trillion-parameters-what-it-means-for-ai?utm_source=hashnode&utm_medium=feed) DeepSeek V4 with 1 trillion parameters is set to reshape AI. Analysis of its open-source impact, technical architecture, and what it means for the AI industry. 6 min read · [wowhow.hashnode.dev](https://wowhow.hashnode.dev/deepseek-v4-1-trillion-parameters-what-it-means-for-ai?utm_source=hashnode&utm_medium=feed) [Anup Karanjkar](https://hashnode.com/@anupkaranjkar) · [May 2](https://hashnode.com/posts/mistral-small-4-open-source-model-replaces-three-apache-2026/69f583247f5d81b9a455252b) ### [Mistral Small 4: One Open-Source Model That Replaces Three (March 2026)](https://wowhow.hashnode.dev/mistral-small-4-open-source-model-replaces-three-apache-2026?utm_source=hashnode&utm_medium=feed) Mistral Small 4 review: open-source Apache 2.0 AI that unifies reasoning, vision, and coding into one MoE model. Benchmarks, features, and deployment guide. 9 min read · [wowhow.hashnode.dev](https://wowhow.hashnode.dev/mistral-small-4-open-source-model-replaces-three-apache-2026?utm_source=hashnode&utm_medium=feed) [Anup Karanjkar](https://hashnode.com/@anupkaranjkar) · [Apr 27](https://hashnode.com/posts/claude-mythos-5-10-trillion-parameter-moe-developer-guide-2026/69efc0473cfb0e9de43df478) ### [Claude Mythos 5: Anthropic's 10-Trillion Parameter MoE Model — Complete Developer Guide (April 2026)](https://wowhow.hashnode.dev/claude-mythos-5-10-trillion-parameter-moe-developer-guide-2026?utm_source=hashnode&utm_medium=feed) Claude Mythos 5 uses 10T parameters (MoE, ~1T active) to set a new frontier. Architecture, $40-60/M pricing, Glasswing access, and a decision framework inside. 10 min read · [wowhow.hashnode.dev](https://wowhow.hashnode.dev/claude-mythos-5-10-trillion-parameter-moe-developer-guide-2026?utm_source=hashnode&utm_medium=feed) [Jangwook Kim](https://hashnode.com/@effloow) · [Apr 24](https://hashnode.com/posts/llama-4-maverick-developer-guide-2026/69eab6ed7d86a71766aa9ed1) ### [Llama 4 Maverick: 400B MoE Model — Self-Hosting and API Guide](https://effloow.hashnode.dev/llama-4-maverick-developer-guide-2026?utm_source=hashnode&utm_medium=feed) \## Why Llama 4 Maverick Matters In April 2025, Meta shipped the Llama 4 family and reset what open-weight models can do. Llama 4 Maverick — the mid-tier model in the three-model lineup (Scout, Maverick, Behemoth) — packs 400 billion total parameters... 9 min read · [effloow.hashnode.dev](https://effloow.hashnode.dev/llama-4-maverick-developer-guide-2026?utm_source=hashnode&utm_medium=feed) [Abstract Algorithms](https://hashnode.com/@abstractalgorithms) · [Apr 17](https://hashnode.com/posts/sparse-mixture-of-experts-llm-architecture/69e26477013bfd2e26f3f753) ### [Sparse Mixture of Experts: How MoE LLMs Do More With Less Compute](https://abstractalgorithms.hashnode.dev/sparse-mixture-of-experts-llm-architecture?utm_source=hashnode&utm_medium=feed) How MoE replaces the dense FFN with N expert layers and a learned router — GPT-4-scale capacity at a fraction of inference cost 27 min read · [abstractalgorithms.hashnode.dev](https://abstractalgorithms.hashnode.dev/sparse-mixture-of-experts-llm-architecture?utm_source=hashnode&utm_medium=feed)