MoE lets Gemini handle 1M tokens efficiently by activating just 2 of 8 experts per token—cutting inference costs while preserving massive model capacity.
From silicon to softmax — ML systems, LLM internals, and research at the edge.
ML/AI student researching disaster-response edge computing. Currently authoring MOSAIC — a Linux userspace scheduler targeting ICCNCom. Writing about LLM internals, systems ML, and everything between the hardware and the model. Targeting Google Research
Markdown supported · ⌘↵ to post
No responses yet.