liveBook · Manning

Evaluation suite

As explained in Unit Testing Principles, Practices, and Patterns

A test suite is a collection of automated tests evaluated together for a software system or component.

A test suite is a collection of automated tests evaluated together for a software system or component. Its purpose is to support sustainable growth by catching regressions, enabling refactoring, and preserving development pace without imposing excessive maintenance cost. section 1.0 section 1.6

Build a successful suite by ensuring that it:

  • Runs as part of the development cycle, ideally after every code change. section 1.4
  • Concentrates on the most important parts of the codebase, especially business logic and the domain model. section 1.4
  • Retains tests whose benefits exceed their implementation and maintenance costs. section 1.6

Coverage is not quality

Treat low coverage as a warning, not high coverage as proof. Branch coverage can reveal more completeness than code coverage, but neither accounts for assertions or proves that the suite protects important behavior. Imposing a target percentage can therefore reward superficial tests. section 1.6

Evaluate tests individually and apply judgment rather than relying on an automated quality score. A valuable suite protects observable behavior while avoiding fragile, low-value tests and unnecessary coverage of less important code. section 1.4 section 4.0

How Unit Testing Principles, Practices, and Patterns covers it

How deep it goes
Defines suite-level quality, prioritization, and evaluation, while deferring detailed criteria for individual tests to a later treatment.
Examples
Contrasts continuous execution, domain-focused targeting, cost-benefit selection, and misleading code or branch coverage targets.
What it assumes
Assumes familiarity with automated testing while treating testing concepts as language-independent across object-oriented languages.
What you can do after
Evaluate and refactor a test suite by weighing each test’s benefits against its maintenance costs and regression protection.
How it connects
Connects automated testing with unit testing, integration testing, end-to-end testing, business logic, coverage metrics, and the Test Pyramid.

Also explained in

Compare how 11 books explain evaluation suite →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Unit Testing Principles, Practices, and Patterns. The section links point to the parts of the book it draws on.