liveBook · Manning

Acceptance testing

As explained in Vibe Engineering

A test that evaluates whether a system or feature satisfies product-level requirements from a user's or stakeholder's perspective.

A green test suite can still approve the wrong behavior. Acceptance testing checks a feature against product requirements, independently of what its implementation appears to do. That independence matters when an agent writes both the code and its tests: both can encode the same misunderstanding. section 10.4.1 section 10.4.2

A rounding error that passes

A test expected a 50% discount on 9.99 to return 4.99, and the code returned 4.99. Both agreed, but the required answer was 5.00. An acceptance test derived from the requirement rather than the implementation can expose that mismatch. section 10.4.1

Agent-generated unit tests can help detect regressions, but they do not independently establish that intended behavior meets product requirements. Use human-written acceptance tests or tests generated from ticket requirements, not from the code, to provide that separate check. section 10.4.1 section 10.4.2

For agent-generated pull requests that change business logic, require at least one human-written or ticket-derived acceptance test. In one team, this added about 15 minutes per pull request and caught three specification violations in the first month. Check test integrity as well: a green CI result can also come from disabling tests or weakening assertions. section 10.4.2 section 10.4.1

How Vibe Engineering covers it

How deep it goes
Explains circular tests and prescribes a human-written or ticket-derived acceptance test for each agent-generated pull request touching business logic.
Examples
Uses a discount-rounding mismatch and a team’s pull-request requirement to illustrate specification violations and their detection.
What you can do after
Practice writing or deriving acceptance checks from requirements before trusting tests for agent-generated business logic.
How it connects
Connects acceptance testing to specifications, unit testing, regression detection, and continuous integration.

Also explained in

Compare how 5 books explain acceptance testing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.