liveBook · Manning

Quality

As explained in Vibe Engineering

Quality is the degree to which a system's outputs satisfy required correctness, usefulness, and user expectations.

Quality is not a free property of AI-assisted development. Teams that cut API costs without tracking output quality can discover the damage only after customers complain, creating a delayed and difficult-to-locate regression. section 9.0

Treat quality as a workflow-dependent tradeoff against latency and cost. A single middle-tier configuration often performs poorly across interactive coding, batch processing, complex reasoning, and lightweight completion. section 9.5.4

Evaluate quality against the task rather than a single global benchmark:

  • Use stronger models for complex reasoning, multi-file generation, deep CI review, and overnight test generation when quality matters more than latency. section 9.5.4
  • Accept lower quality for simple, short tasks such as tab completion, classification, boilerplate, and immediate first-pass CI review. section 9.5.4 section 9.3.5
  • Test quantized models on representative work: degradation may be negligible for short completions but visible in multi-line functions and test suites. section 9.3.5

Choosing a local coding model

A team comparing Q4_K_M with Q6_K found less than a 2% acceptance-rate difference on tab completion, inline edits, and test scaffolding, while Q4_K_M preserved more RAM for the IDE and containers. For function-length generation, it stayed at Q4_K_M or above because lower levels produced inconsistent names, subtle control-flow bugs, and forgotten constraints. section 9.3.5

Table 9.7 Matching model to the usecase
Technique
Properties
Model
Interactive pair-programming
Latency + Quality
Sonnet, agent mode
Tab completion
Cost + Latency
Haiku or local model
Background refactoring
Cost + Quality
Sonnet, async/batch
CI review (first pass)
Cost + Latency
Haiku, immediate
CI review (deep pass)
Cost + Quality
Sonnet, batch API
Test generation
Cost + Quality
Sonnet, overnight
Documentation
Cost + Latency
Haiku, on-demand

Use the workflow matrix to match quality requirements with latency and cost instead of applying one model configuration everywhere. section 9.5.4

How Vibe Engineering covers it

How deep it goes
Compares workflow and quantization tradeoffs, but stops short of defining a universal quality metric or evaluation protocol.
Examples
Uses customer complaints, model-workflow assignments, quantization levels, acceptance rates, completions, functions, and generated test suites.
What it assumes
Assumes AI-assisted development and production cost concerns, while introducing the latency-cost-quality tradeoff explicitly.
What you can do after
Evaluate model configurations against representative workflows and choose where to spend latency, cost, or quality.
How it connects
Connects quality to latency, cost optimization, model selection, quantization, code generation, and continuous integration review.

Also explained in

Compare how 10 books explain quality →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.