liveBook · Manning

Model family

As explained in Rearchitecting LLMs

A model family is a set of related models arranged or designed to provide different capabilities or resource requirements.

A model family uses related models with different capabilities and resource requirements to handle agent tasks without sending every request to a frontier model. Narrow, repetitive tool-routing decisions can be handled by smaller specialists, reducing the cost and latency of repeated calls. section 13.0

A cascaded family derives each smaller model from the one before it:

  • T (Teacher) is a specialist fine-tuned from Qwen3-0.6B. section 13.1
  • M (Medium) is pruned and distilled from T; S (Small) is pruned and distilled from M. section 13.1
  • The resulting family can support speculative decoding and difficulty-based routing. section 13.0
Figure 13.1 The model family cascade. Box height represents the number of transformer blocks, box width represents the MLP expansion size, and the narrow column represents the hidden dimension, which stays fixed at 1024 across the cascade. Each model is created from the previous one through pruning and knowledge distillation, and acts as its teacher. M is the default teacher for S; T can be used as an alternative teacher, with pruning always applied to M regardless of which teacher is chosen.

The cascade diagram shows how model size changes across T, M, and S while the hidden dimension stays fixed at 1024. M is the default teacher for S; T can be used as an alternative teacher. section 13.1

Weather-query agent

Answering “What’s the weather like where I am?” involves three tool calls: obtaining a postal code from an IP address, converting it to a city, and checking the weather. This illustrates a sequence of narrow decisions that need not all use a frontier model. section 13.1

Evaluate each model against the same baseline tests, hardware, and configuration. Compare task and general capabilities alongside inference efficiency, and choose a production model against an acceptance criterion set in advance. section 13.7

How Rearchitecting LLMs covers it

How deep it goes
Explains a fine-tuned teacher and sequential pruning and distillation, then outlines baseline and post-change evaluation.
Examples
Uses a weather-query agent with three tool calls and a T–M–S cascade based on Qwen3-0.6B.
What you can do after
Build it from scratch by fine-tuning a specialist, deriving smaller models, and comparing identical capability and efficiency tests.
How it connects
Connects model families to fine-tuning, pruning, knowledge distillation, speculative decoding, and difficulty-based routing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.