For most of the last decade, AI models got better in the most brute-force way imaginable: by getting bigger. More parameters, more data, more compute, more cost. But there’s a smarter question hiding inside that arms race — do you really need to use the whole model for every single word? Mixture of Experts (MoE) is the architecture that answers “no,” and it’s a big part of why models like DeepSeek can be enormous and efficient at the same time.
If you’ve ever wondered how a model with 671 billion parameters can run without a supercomputer melting, the answer is that it never uses all 671 billion at once. This guide explains what Mixture of Experts is, how it works, and why it quietly became one of the most important ideas in modern AI — without any math beyond counting.
The Core Idea: Don’t Wake the Whole Company for One Question
Picture a large consulting firm with hundreds of specialists — tax lawyers, engineers, designers, translators. When a client sends in a question about a bridge, you don’t drag all 300 people into the room. You route the question to the two or three engineers who can actually answer it. Everyone else stays at their desk.
A traditional (“dense”) neural network does the opposite. For every single word it processes, it runs every parameter in the model — the whole firm, every time. That’s simple, but wildly wasteful once the model gets big. Mixture of Experts fixes this by splitting part of the network into many smaller sub-networks called experts, and adding a lightweight router that decides, for each word, which few experts should handle it.
The result is the single most important idea in this whole topic: activated parameters are not the same as total parameters. DeepSeek-V3 stores 671 billion parameters, but only activates about 37 billion — roughly 5.5% — for any given word. The knowledge is all there; only a small, relevant slice wakes up at a time.
How the Routing Actually Works
Inside an MoE layer, three things happen for each word (token):
- The router scores the experts. A small, fast component looks at the token and produces a score for each expert — essentially, “how well does each specialist match this input?”
- Only the top few are chosen. Instead of using all experts, the model keeps just the highest-scoring handful and ignores the rest. This is the “sparse” in sparsely-activated — most of the model sits idle for this token.
- Their answers are combined. The chosen experts each process the token, and their outputs are blended (weighted by those router scores) into the final result.
The next token might get routed to a completely different set of experts. Over billions of tokens, different experts drift toward different kinds of knowledge — one might get good at code, another at a particular language, another at formatting — without anyone explicitly assigning them those jobs. It emerges from the routing.
DeepSeek’s Recipe, by the Numbers
DeepSeek is a great concrete example because its makers published the exact configuration. Each of its MoE layers has:
- 256 “routed” experts — the big pool the router chooses from.
- 1 “shared” expert — one specialist that is always on, for every token, no matter what.
- Top-8 routing — the router picks the 8 best-matching routed experts for each token.
So for any given word, 9 of 257 experts fire (the 8 chosen ones plus the always-on shared expert). That’s exactly how 671 billion total parameters collapse to ~37 billion active. (Not every layer works this way — the first 3 of DeepSeek’s 61 layers are ordinary dense layers, a stability choice; the other 58 are MoE.)
That shared expert is a clever touch. Because it sees every token, it soaks up the common, general-purpose knowledge every word needs — grammar, basic structure, common sense — which frees the 256 routed experts to specialize more cleanly instead of each re-learning the basics. It’s like having one generalist office manager who handles the routine paperwork for everyone, so the specialists only get pulled in for what they’re actually best at.
MoE Did Not Start With DeepSeek
It’s important to be honest about lineage here, because a lot of 2025 coverage implied DeepSeek invented this. It didn’t.
Sparsely-gated Mixture of Experts for neural networks was introduced by Shazeer and colleagues in 2017. Google scaled it up for translation with GShard (2020), then pushed it into trillion-parameter territory with the Switch Transformer (2021). Mistral’s Mixtral 8×7B (2023) made MoE famous in the open-weights world, and GPT-4 has long been rumored (never confirmed) to use it.
DeepSeek’s genuine contributions are the specific variant — many fine-grained experts plus the always-on shared expert — and executing it well at 671-billion-parameter scale. That’s a real achievement, but it’s an evolution of a well-established idea, not a bolt from the blue. Knowing that helps you read AI news more critically: “efficient new architecture” is often a smart refinement of something a few years old.
Why Mixture of Experts Matters
MoE reshaped the economics of large models in two directions at once:
Cheaper to run. Because only a fraction of the model activates per token, an MoE model generates text using far less compute than a dense model of the same total size. That directly lowers the cost of every answer — part of the reason DeepSeek’s efficiency story made headlines (though that “$6 million” figure needs an asterisk).
But not cheaper to store. Here’s the catch people miss: activated parameters are a compute measure, not a memory one. Even though only 37 billion parameters fire per token, all 671 billion still have to be loaded and kept in memory, because the router might call on any of them next. That’s why a “37B active” model absolutely will not run on your laptop — it still needs the hardware to hold the full 671 billion. Efficiency in compute doesn’t mean efficiency in storage.
That distinction matters for anyone comparing model spec sheets: a headline parameter count tells you how much knowledge a model holds, but the activated count tells you how much work it does per word. With MoE, those two numbers can differ by more than 10×.
Mixture of Experts and the Bigger Picture
MoE is one of several efficiency tricks that let modern large language models scale without costs exploding — it works alongside the attention mechanism at the heart of every transformer. It also introduces its own new problem: if the router keeps favoriting a few popular experts and starving the rest, you waste all that specialized capacity. Solving that — keeping the workload balanced across experts without hurting quality — is an engineering art in itself, and one DeepSeek handled with a neat trick we cover in the main DeepSeek explainer.
The big takeaway is simple: the future of large AI isn’t only about making models bigger. It’s about being smart about which parts of a big model you actually use. Mixture of Experts is the clearest expression of that idea — a giant brain that only ever thinks with the part it needs.
❓ Frequently Asked Questions
What is Mixture of Experts in simple terms? Mixture of Experts (MoE) is an AI model design that splits part of the network into many smaller “expert” sub-networks and uses a router to send each word to only a few of them, instead of running the entire model every time. It lets a model store a huge amount of knowledge while only using a small, relevant slice for any given word — making it much cheaper to run.
How is MoE different from a normal (“dense”) model? A dense model uses every one of its parameters for every token. An MoE model activates only a small subset — for example, DeepSeek uses 9 of its 257 experts per layer per token. Same idea as routing a question to the right specialists instead of assembling the whole company.
Does Mixture of Experts make a model smaller? No — it makes it cheaper to run, not smaller to store. All the parameters still have to be kept in memory because the router might need any of them next. DeepSeek activates ~37 billion parameters per token but still requires all 671 billion to be loaded, which is why big MoE models still need serious hardware.
Did DeepSeek invent Mixture of Experts? No. Sparsely-gated MoE dates to Shazeer et al. in 2017, and was scaled by Google’s GShard and Switch Transformer and by Mistral’s Mixtral. DeepSeek’s contribution is its particular fine-grained-plus-shared-expert design at very large scale — a refinement, not the original invention.
Why don’t all AI models use MoE? MoE adds complexity: the router has to be trained well, the experts have to be kept balanced so none are overused or ignored, and the full model still demands a lot of memory. For smaller models the overhead isn’t worth it — MoE shines mainly at very large scale, where the compute savings become dramatic.
📚 Keep Learning
- Learn AI and ML Concepts Through the Lens of DeepSeek — the full tour, including how DeepSeek keeps its experts balanced.
- How Large Language Models Work and How Transformer Attention Works — the foundations MoE is built on top of.
- Did DeepSeek Really Cost $6 Million? — how this efficiency translates (and doesn’t) into cost.
📚 Recommended Resources
Disclosure: some links below are Amazon affiliate links. If you buy through them, I earn a small commission — at no extra cost to you — which helps keep this site running.
Deep Learning with Python
François Chollet · Book · ~$50 on Amazon
The friendliest on-ramp to how neural networks actually work, from the creator of Keras. If this article made you curious about what “experts” and “layers” really are under the hood, this is the clearest place to build that intuition — with runnable code, minimal jargon.
Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow
Aurélien Géron · Book · ~$70 on Amazon
The single best structured reference for going from fundamentals to modern deep learning. When you’re ready to move past analogies and see how routing, layers, and training really fit together, this is the book to keep on your desk.
Transformers for Natural Language Processing
Denis Rothman · Book · ~$50 on Amazon
Mixture of Experts is a variation on the transformer architecture, so if you want to understand the machine MoE plugs into, start here. A practical, applied tour of how modern language models are built and used.