liveBook · Manning

Model selection

As explained in Vibe Engineering

Model selection is the choice of a particular AI model or service tier for a task based on capability, cost, and operational requirements.

Model selection matches an AI model or service tier to a task’s quality, latency, cost, and operating requirements. A cheaper tier can reduce the visible bill while increasing retries, developer wait time, or downstream quality problems. Evaluate the workflow’s total cost, not token price alone. section 9.6 section 9.1.3

Choose for the workflow rather than imposing one global setup:

  • Pair programming favors latency and quality; CI review favors cost and quality; tab completion favors cost and latency. section 9.6
  • For local coding, prefer a coding-specific model when available. Local selection is constrained by what fits in memory, not just by the model’s advertised capability. section 9.3.4
  • Consider local inference for offline work, privacy-sensitive codebases, and experimentation, even when cost savings are not the main benefit. section 9.6

A mixture-of-experts model makes local sizing less intuitive: only some parameters are active for each token, but all weights must reside in memory. A model that fits by weight size alone may still exceed available memory once the context and KV cache are included. section 9.3.4 section 9.3.4

Sizing local coding work

Under 16 GB, small local models are suited to boilerplate, imports, and pattern continuation, but struggle with reasoning across files. Around 24–32 GB, inline edits and single-file work improve, while multi-file reasoning remains less reliable. At 48 GB and above, larger mixture-of-experts models can support multi-file agent work, subject to context memory requirements. section 9.3.4

Treat a tier swap as a behavior change, not just a configuration edit. Budget for the swap, a review for regressions, and a canary with real traffic. Track first-attempt success and sample results weekly so a gradual quality decline does not erase apparent savings. section 9.1.3 section 9.6

Before starting an optimization project, compare likely savings with engineering time. A rule of thumb is to avoid optimizing when monthly AI tooling spend is below two weeks of one engineer’s fully loaded cost. Improving existing tool use and prompt habits may be the cheaper first step. section 9.1.3

How Vibe Engineering covers it

How deep it goes
Compares workflow tradeoffs and local memory thresholds, then recommends regression review and real-traffic canaries for tier changes.
Examples
Uses tab completion, inline edits, multi-file agent work, pair programming, and CI review to contrast requirements.
What you can do after
Evaluate model choices using task requirements, total cost, memory limits, and observed quality.
How it connects
Connects model selection to local inference, mixture-of-experts models, KV cache, and quality monitoring.

Also explained in

Compare how 12 books explain model selection →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.