liveBook · Manning

Acceptance rate

As explained in Vibe Engineering

Developer acceptance rate is the proportion of AI-generated suggestions or outputs that developers accept for use.

Acceptance rate measures the share of AI-generated suggestions or outputs that developers accept. In model comparisons, acceptance means using output without significant modification. section 9.2.4

Treat it as a workflow-specific quality signal, not a universal score: measured rates vary across teams, tasks, prompts, and codebases. Lower rates can also mean more retries or manual corrections. section 9.1.3 section 9.2.4

Use the metric to guide engineering decisions:

  • Compare models by task: trivial suggestions showed little differentiation, while complex debugging and multi-file work showed larger gaps. section 9.2.4
  • Check context practices: providing relevant files and clear instructions was associated with higher acceptance than supplying broad, tangential context. section 9.2.3
  • Include human rework in cost decisions; cheaper output can cost more overall when developers retry or override it more often. section 9.1.3
  • Measure local model changes on representative tasks: one 500-completion test found less than a 2% difference between two quantization levels. section 9.3.5
Table 9.1 Change of costs between models
Task Type
Sonnet
Haiku
GPT-4.1-mini
 
Model to be selected
Tab completion
94%
93%
92%
Any (≈)
Import statements
97%
96%
96%
Any (≈)
Boilerplate generation
91%
88%
87%
Any (≈)
Unit test scaffolding
89%
82%
80%
Sonnet*
Inline refactoring
90%
83%
81%
Sonnet*
Function implementation
88%
74%
72%
Sonnet
Complex debugging
85%
61%
58%
Sonnet
Multi-file refactoring
82%
52%
49%
Sonnet
Architecture planning
79%
41%
38%
Sonnet

Compare reported acceptance rates across model tiers and coding task types. section 9.2.4

How Vibe Engineering covers it

How deep it goes
Defines acceptance operationally and compares task-level results, while warning that percentages vary by team, codebase, and workflow.
Examples
Compares acceptance across coding tasks and models, cites a team decline from 80% to 50%, and reports a sub-2% quantization difference.
What you can do after
Evaluate coding-assistant output by comparing accepted-without-significant-modification rates across tasks, models, and configurations.
How it connects
Connects model selection, context engineering, quantization, and code review to measured developer acceptance.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.