🧩 Dense vs mixture-of-experts: what it means for your hardware bill
MoE models activate a fraction of their parameters per token. That changes compute and latency — but not memory, which is where naive sizing goes wrong.
A mixture-of-experts (MoE) model replaces some feed-forward layers with a set of "experts" and a router that sends each token to a few of them. DeepSeek-V3 has 671B total parameters but activates about 37B per token; Qwen3-235B-A22B activates 22B. The question for infrastructure is which number matters where.
Compute and speed follow active parameters
Decode throughput and latency scale with the parameters actually touched per token, so a 37B-active MoE runs at roughly the speed of a 37B dense model. That is the headline benefit: frontier-class quality at mid-size speed.
Memory follows total parameters
Every expert must be resident in GPU memory, because the router can choose any of them. A 671B-parameter model therefore needs ~400 GB of weights even at INT4 — an 8-GPU node, not a single card. This is the trap: sizing on "37B active" and buying one H100.
When MoE wins
- You already have (or can rent) a multi-GPU node and need maximum quality per token of latency.
- High-throughput serving where per-token cost matters more than fixed memory cost.
When dense wins
- Single-GPU or edge deployments where total memory is the constraint.
- Small teams that want simple tensor-parallel setups without expert-parallel tuning.
The catalog labels each model dense or MoE and lists both total and active parameters; the sizing calculator uses total parameters for memory and active parameters for speed.