A router directs work to a processing path based on the input or a runtime decision. At the request level, it can select an adapter queue or escalate an uncertain answer to an API. Inside a mixture of experts (MoE), it selects or weights experts for each token; this is a different routing scope from directing whole requests. section 7.3.6 section 12.8.2 section 9.1
For multiple model specializations, classify incoming requests by task and buffer them in separate queues. Process a batch with its adapter active, then switch adapters for the next batch while keeping the base model in memory. Switching on every request can erase the performance benefit; the choice between swappable and merged adapters depends on specialization count, switching frequency, and hardware. section 7.3.6
Figure 7.7 Adapter routing pattern for multiple specializations over a single base model. A task router classifies incoming requests and buffers them into separate queues. A switch selects which adapter is active: the model processes one full batch, swaps the adapter, and processes the next. The base model stays in memory throughout — only the lightweight adapter changes between batches. The dashed outline indicates the inactive adapter waiting for its turn.
The diagram shows task queues feeding a shared base model, with one adapter active per batch. section 7.3.6
A confidence-based router can instead run a small language model first and send uncertain cases to the original API. One uncertainty signal is the probability of the least confident generated token. Tune the escalation threshold against quality and cost: too much fallback can consume the savings. Before cutover, run both paths in shadow, serve the API response, and log disagreements; then introduce the small model behind a limited canary. section 12.8.2
An MoE router operates on contextual hidden states, not raw tokens. A linear layer projects each token's hidden state to one logit per expert. Soft routing combines expert outputs using routing weights; in the two-expert implementation, both experts run, so specialization does not itself reduce active computation. Top-k routing instead selects the highest-scoring experts per token and normalizes their weights before combining their outputs. section 9.2.1 section 9.2.2 section 9.3.1 section 9.6
Figure 9.4 Two views of the same top-k routing decision. Left: the router produces one logit score per expert for each token; the two highest scores per row (highlighted) are selected. Right: the same assignments reorganized by expert, showing which tokens each expert will process and their renormalized weights — for each token, those weights sum to 1 across its assigned experts. The example uses four experts rather than two so that top_k = 2 leaves half the experts idle per token. The loop in TrainableTopKMoE.forward performs exactly this regrouping.
The top-k diagram connects per-token expert scores to the regrouping of tokens by expert. section 9.3.1
Training and checking a two-expert router
Replace selected SmolLM2 MLPs with blocks containing the frozen original MLP, a trainable clinical copy, and a trainable router. Inspect stored routing weights after a forward pass: a general-knowledge prompt mildly favors the general expert, while several tokens in a clinical note favor the clinical expert. These token-level checks complement task evaluation; they do not alone establish routing quality. section 9.2 section 9.2.2
A pretrained clinical MLP can also be transplanted as an expert. Router-only training with both experts frozen reached 90% schema compliance on the test set; after training the router and the last six clinical expert layers for one more epoch, compliance reached 100% with both top_k=1 and top_k=2. The additional training adapts the expert to the base model's attention layers. Conditional routing may require inference-engine adaptations, so validate deployment compatibility as well as task quality. section 9.3 section 9.3.2 section 9.3.3 section 9.1

