liveBook · Manning

Evaluation suite

As explained in BDD in Action, Second Edition

A test suite is a collection of tests organized to evaluate software behavior against defined expectations.

A team cannot manually recheck every acceptance criterion after every change without slowing feedback. A test suite collects automated checks of expected behavior so changes can be assessed repeatedly. When those checks are executable specifications, they also show how the application meets business requirements in language the team can share. section 2.3.6

The suite’s value depends on its design, not its size. Treating tests as disposable scripts can produce brittle checks whose maintenance costs outweigh their feedback. A useful acceptance test suite has passing or pending tests; a failure calls for prompt attention rather than becoming another routinely broken build result. section 8.2 section 9.1

In a Java Cucumber project, keep the parts of the suite distinct:

  • Put executable specifications and test code under src/test, separate from application code under src/main. section 8.2.2
  • Store feature files in resources/features, grouping them by capability when one directory becomes crowded. section 8.2.2
  • Use a test runner to execute feature files and step definitions to connect scenario steps to code. section 8.2.2

From a shared example to a check

A scenario step such as “Given Tess has a current account with $1000” states a starting condition in business terms. A Cucumber step definition matches the step, extracts the account type and balance, and sets up the account. Run with the build, the scenario can check a new feature and later guard against regression. section 2.3.6

For a growing system, design more than a collection of scenarios. The suite operates within a test automation system that includes frameworks, libraries, and glue code. Keeping business scenarios distinct from low-level interaction code helps the suite remain understandable and useful for verifying existing behavior and test-driving new functionality. section 14.2

How BDD in Action, Second Edition covers it

How deep it goes
Explains suite organization, reliability concerns, and its supporting automation system, with a concrete Cucumber project layout.
Examples
Uses a bank-account Cucumber step, a Java project layout, and frequent-flyer scenarios.
What it assumes
Starts with project setup from scratch before developing test code through Java and JavaScript examples.
What you can do after
Build it from scratch as an organized acceptance test suite that gives reliable feedback and supports future functionality.
How it connects
Connects test suites to acceptance criteria, executable specifications, regression testing, step definitions, and test automation systems.

Also explained in

Compare how 11 books explain evaluation suite →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of BDD in Action, Second Edition. The section links point to the parts of the book it draws on.