A gating network, also called an expert router, is the small learned component in a mixture of experts architecture that assigns each input to one or more expert subnetworks and sets how strongly each selected expert contributes to the output. In neural networks, it is the mechanism that makes conditional computation possible: the model holds many parameters but activates only a fraction of them for any given token or example.
Within Artificial intelligence research, gating networks became central to large language models because they allow parameter count to grow faster than compute cost. A sparse model with dozens of experts can match the quality of a much larger dense model while spending roughly the floating-point operations of a single expert per token.
How a gating network routes inputs
A gating network computes a score for every expert from the input representation and converts those scores into routing weights. In the most common form, the router is a single linear layer that multiplies the hidden state of a token by a learned router matrix, producing one logit per expert, followed by a softmax.
In sparse routing, only the top-k experts with the highest scores are kept, typically k equal to 1 or 2, and their weights are renormalized. The layer output is the weighted sum of the selected experts’ outputs. Because the gating weights are differentiable, gradients flow back into the router, so it learns which experts suit which inputs during ordinary training.
The table below summarizes the steps that happen inside a mixture of experts layer for each token.
| Step | Operation | Result |
|---|---|---|
| Scoring | Linear projection of the token state | One logit per expert |
| Normalization | Softmax or sigmoid over logits | Routing probabilities |
| Selection | Keep the top-k scores | A small active expert set |
| Dispatch | Send the token to chosen experts | Expert-specific computation |
| Combination | Weighted sum of expert outputs | Layer output for the token |
Routing strategies and their parameters
Gating networks differ mainly in how many experts they activate and who chooses whom. The choice changes model quality, training stability and hardware efficiency, so it is one of the most consequential design decisions in a sparse model.
| Strategy | How it selects | Strength | Weakness |
|---|---|---|---|
| Dense soft gating | All experts, weighted by softmax | Smooth, stable training | No compute savings |
| Top-1 routing | Single best expert per token | Lowest compute and communication | Higher risk of imbalance |
| Top-2 routing | Two best experts per token | Better quality, more robust | About twice the expert compute |
| Expert-choice routing | Each expert picks its top tokens | Perfect load balance by design | Some tokens may be skipped |
| Noisy top-k gating | Adds tunable noise before selection | Encourages exploration of experts | Extra hyperparameters |
Several numeric settings control the router. The capacity factor, usually between 1.0 and 1.25 in training, sets how many tokens each expert may accept per batch; tokens beyond that limit are dropped or passed through the residual connection. The number of experts commonly ranges from 8 to several hundred per layer, and some designs add shared experts that every token uses alongside the routed ones.
Keeping expert load balanced
The central difficulty of a gating network is routing collapse, a self-reinforcing loop in which a few experts receive most tokens, train faster, and therefore attract even more tokens. Unused experts stay undertrained, and the model loses the capacity it was designed to have.
Engineers counter this with auxiliary load-balancing losses that penalize uneven token distribution, a router z-loss that keeps router logits small for numerical stability, and bias-based balancing that adjusts per-expert offsets without adding a loss term. Each approach trades a little modeling freedom for stable, evenly used experts.
| Symptom | Likely cause | Typical remedy |
|---|---|---|
| Few experts receive most tokens | Routing collapse | Auxiliary balancing loss or bias adjustment |
| Many tokens dropped | Capacity factor too low | Raise capacity or use dropless kernels |
| Loss spikes during training | Large router logits | Router z-loss, higher precision router |
| Experts learn redundant skills | Weak specialization pressure | Fine-grained experts plus shared experts |
Gating network compared with related mechanisms
A gating network is easily confused with other uses of the word gate in neural networks, yet its role is distinct. It selects which subnetwork runs, whereas most other gates control how much information passes within a single fixed computation.
| Mechanism | What it controls | Granularity |
|---|---|---|
| Gating network | Which experts process an input | Per token or per example |
| LSTM and GRU gates | Flow of memory through a recurrent cell | Per hidden unit and time step |
| Attention mechanism | Weighting of other tokens’ values | Per token pair |
| Hypernetwork | Generation of another network’s weights | Per task or input |
The attention mechanism and the gating network both use softmax over learned scores, but attention mixes information between tokens, while the router mixes computation paths for one token. The mixture of experts term describes the whole architecture; the gating network is only its decision-making part.
Practical limitations and common mistakes
Sparse routing reduces arithmetic but not memory: all experts must be stored, so a model with a modest active parameter count can still require many accelerators to serve. Routing also introduces all-to-all communication when experts sit on different devices, which can dominate step time on slow interconnects.
- Assuming experts are topic specialists: analyses often show that routers specialize on token types or syntax rather than human-readable subjects.
- Ignoring inference batching: small batches leave many experts idle, so throughput gains seen in training may not appear in serving.
- Fine-tuning the router carelessly: aggressive updates on narrow data can destabilize a well-balanced pretrained router; many teams freeze or regularize the router during adaptation.
In simple terms
A gating network works like a receptionist in a large clinic. Each patient arrives, the receptionist looks briefly at the problem, and sends the patient to the one or two specialists most likely to help. The other doctors stay free for other patients.
In an AI model, the patients are pieces of text and the specialists are small expert networks. Because only a few experts work on each piece, the model can be very large yet still fast. The difficult part is making sure the receptionist does not send everyone to the same doctor.
A practical example
A research team trains a language model with 64 experts per layer and top-2 routing, so each token activates only about one thirty-second of the expert parameters. Early in training, monitoring dashboards show that four experts receive nearly half of all tokens while a dozen experts receive almost none, and the training loss starts to plateau.
The team adds an auxiliary load-balancing loss with a small coefficient, introduces a router z-loss, and raises the capacity factor from 1.0 to 1.25. Within a few thousand steps the token share per expert settles close to the ideal average, dropped tokens fall below one percent, and validation loss resumes its decline. The final sparse model reaches quality comparable to a dense model with several times more active compute per token.