liveBook · Manning

Mixture of experts

As explained in Vibe Engineering

A mixture-of-experts model is a neural-network architecture that selectively activates a subset of specialized parameter groups for each input.

A mixture-of-experts (MoE) model divides parameters among specialists and activates only a few for each token. This reduces per-token computation without reducing the memory required to keep the full model resident: an 80B model with 3B active parameters can approach 3B-model speed while consuming 80B-model memory. section 9.3.4

Local MoE deployment is governed by hardware thresholds:

  • Below 16GB, local models mainly support tab completion, boilerplate, imports, and pattern continuation. section 9.3.4
  • At roughly 24–32GB, tab completion, inline edits, and single-file work become useful, while multi-file reasoning remains unreliable. section 9.3.4
  • At 48GB or more, large MoE models can support multi-file agent work, but long contexts add KV-cache memory costs. section 9.3.4

A large model with small active compute

Qwen3-Coder-Next is described as an 80B MoE model that activates 3B parameters per token, achieves genuine multi-file capability on a 48GB unified-memory Mac, and still trails frontier models on the hardest cross-cutting tasks. section 9.3.1

How Vibe Engineering covers it

How deep it goes
Explains sparse activation, resident memory, active computation, and KV-cache limits, but does not detail expert-routing internals.
Examples
Uses 80B/3B-active models, Qwen3-Coder-Next, and hardware thresholds from 16GB to 48GB-plus.
What it assumes
Assumes familiarity with local inference, coding models, and GPU or unified-memory constraints.
What you can do after
Operate local inference by matching MoE model size and context demands to available memory.
How it connects
Connects local inference with coding models, latency, RAM, GPU memory, unified memory, and KV cache.

Also explained in

Compare how 10 books explain mixture of experts →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.