Model selection matches an AI model or service tier to a task’s quality, latency, cost, and operating requirements. A cheaper tier can reduce the visible bill while increasing retries, developer wait time, or downstream quality problems. Evaluate the workflow’s total cost, not token price alone. section 9.6 section 9.1.3
Choose for the workflow rather than imposing one global setup:
- Pair programming favors latency and quality; CI review favors cost and quality; tab completion favors cost and latency. section 9.6
- For local coding, prefer a coding-specific model when available. Local selection is constrained by what fits in memory, not just by the model’s advertised capability. section 9.3.4
- Consider local inference for offline work, privacy-sensitive codebases, and experimentation, even when cost savings are not the main benefit. section 9.6
A mixture-of-experts model makes local sizing less intuitive: only some parameters are active for each token, but all weights must reside in memory. A model that fits by weight size alone may still exceed available memory once the context and KV cache are included. section 9.3.4 section 9.3.4
Sizing local coding work
Under 16 GB, small local models are suited to boilerplate, imports, and pattern continuation, but struggle with reasoning across files. Around 24–32 GB, inline edits and single-file work improve, while multi-file reasoning remains less reliable. At 48 GB and above, larger mixture-of-experts models can support multi-file agent work, subject to context memory requirements. section 9.3.4
Treat a tier swap as a behavior change, not just a configuration edit. Budget for the swap, a review for regressions, and a canary with real traffic. Track first-attempt success and sample results weekly so a gradual quality decline does not erase apparent savings. section 9.1.3 section 9.6
Before starting an optimization project, compare likely savings with engineering time. A rule of thumb is to avoid optimizing when monthly AI tooling spend is below two weeks of one engineer’s fully loaded cost. Improving existing tool use and prompt habits may be the cheaper first step. section 9.1.3











