Chapter 12: Getting ready for scenario automation
This introduction explains why scenario automation can be risky and clarifies how the book’s tool-specific examples relate to its largely language-independent patterns.
After this chapter you will be able to
- Risks of scenario automation. Explain why automating more testing does not necessarily deliver the expected benefits.
- Temporary automation quality debt. Explain why delaying scenario automation creates quality debt and when taking on that debt briefly can be justified.
- Focused automation proof-of-concept. Describe how the WIMP team uses a proof-of-concept to gain confidence in its automation solution before automating the remaining scenarios.
- Reusable automation solutions. Explain what a software design pattern is and why reusing a suitable solution to a recurring automation challenge helps keep automation consistent.
- Useful pattern documentation. Describe what a documented automation pattern needs to communicate, regardless of the format used.
- Context-sensitive pattern selection. Justify why a team should check a pattern’s intent, context, constraints, and benefit rather than apply patterns as a shopping list.
- Scenario Text and Automation Code. Explain how scenario automation tools use formulated scenarios to verify expected behavior, and distinguish what the tool does from what automation coders must provide.
- Executable Scenarios and Run Scope. Describe how ordinary scenarios and scenario outlines become executable tests, and explain what feedback comes from running all, some, or one scenario.
- Step definition registration differences. Distinguish how Cucumber and Reqnroll use Given/When/Then keywords when matching scenario steps to registered expressions.
- Reusable step definition workflow. Explain what a step definition is and how a scenario automation tool uses one for each scenario step to compose an executable test. Describe why definitions can be reused across scenarios rather than writing separate automation logic for every test case.
- Step registration and matching. Trace how a registered step keyword and text expression connect scenario text to a step definition, and distinguish the errors caused by no match and multiple matches.
- Expression Matching and Parameters. Explain how a Cucumber Expression matches step text and how placeholders let one step definition handle different test data. Describe how matched values become arguments, including their order when there are multiple placeholders.
- Associated Step Data Arguments. Explain how docstrings and data tables relate to expression matching and how their values are passed to a step definition.
- Custom parameter type purpose. Explain how a custom parameter type extends Cucumber Expression placeholders by converting captured text into a value for a step definition.
- Reqnroll transformation registration. Describe the requirements for a Reqnroll transformation method, including its binding class, placeholder name, and regular expression.
- Expanded step definition matching. Explain that optional or alternate text and custom parameter types can expand a step definition expression into multiple possible matches, all of which are checked against step text.
- Undefined and ambiguous matches. Distinguish what happens when no step definition matches a step from what happens when more than one matches.
- Hook purpose and timing. Explain that hooks run automation logic not tied to a particular step, and describe when scenario, test-run, and step hooks execute.
- Glue Code and Global Availability. Explain what counts as glue code and what its usual global scope means for scenario execution.
- Reusable Glue Code Organization. Describe how the section recommends grouping step definitions and support code, and justify why step-definition folders should not strictly mirror feature files.
- Scenario setup and feedback. Explain how shortcuts, isolated scenarios, and fast feedback contribute to scenario automation.
- Diagnostics and automation code quality. Describe the principles of understandable diagnostics, test-code quality, and atomic assertions.
- Business language and loose coupling. Explain what the business-terminology and loose-coupling principles call for.
- Shortcuts for scenario setup. Explain what an automation shortcut skips and how the team can choose a suitable way to establish a scenario’s required starting state, using the Rebecca example.
- Understandable failure diagnostics. Explain why scenario automation and the system should provide meaningful error messages and test logs when scenarios fail.
- Independent scenario execution. Explain why each automated scenario must have an execution context unaffected by other scenarios, including the state of external services and resources when scenarios run in parallel.
- Test Automation Code Quality. Justify treating test automation code as at least as important as production code by describing its roles beyond guiding implementation.
- Atomic assertion principle. Explain why sequential assertions can leave a failure incompletely diagnosed and how a single atomic assertion can report multiple verification results together.
- Consistent business domain terminology. Explain why named abstractions help people reason about complex systems and why the same business-domain terms should be used from high-level specifications to low-level diagnostic output.
- Loose coupling for automation. Explain how well-defined interfaces let automation coders choose where to initiate behavior and substitute dependencies, and why those choices help them verify behavior efficiently and thoroughly.
- Scenario Driven Development Cycle. Trace the cycle of specifying unimplemented behavior in a scenario, automating it and seeing it fail, then implementing enough functionality to make it pass without breaking existing scenarios.
- Fast Feedback and Diagnosis. Explain why small BDD steps need fast, frequent feedback from the build process and stakeholders, and how a short test suite enables full-suite runs on every commit and helps locate failures.
- BDD and automation strategy. Explain why BDD does not prescribe how scenarios must be automated. Describe what scenario automation tools provide and what the team must still decide.
- Application interface trade-offs. Distinguish inner, outer technical, and UI interfaces for scenario automation, and explain the trade-off between fast inner-interface tests and coverage of the external context.
- Real infrastructure versus stubs. Explain how to choose between real external infrastructure and stubs for scenario automation, and what additional checks may be needed when stubs are used.
What it builds on
- Shared understanding (chapter 6) · Formulated scenarios express a shared understanding of expected behavior.
- Scenario outlines and examples tables (chapter 8) · Scenario outlines use examples tables with rows of data.
- Data tables (chapter 8) · Data tables associated with scenario steps.
- Readable blocks of text (chapter 8) · Multi-line blocks of text in scenarios.
- Structuring the living documentation (chapter 10) · The organization of living documentation, which need not match the organization of step definitions.
- Keep your scenarios BRIEF (chapter 7) · The BRIEF principles for concise, intention-revealing scenarios
- Automation (chapter 1) · The role of automation in BDD.
- Incremental specification (chapter 9) · Incremental specification through small steps.
Section by section
Introduction
This introduction explains why scenario automation can be risky and clarifies how the book’s tool-specific examples relate to its largely language-independent patterns.
- Risks of scenario automation: Explain why automating more testing does not necessarily deliver the expected benefits.
Why it matters: Recognizing these risks helps explain why automation should be approached with care rather than treated as an automatic improvement. - Tool differences and pattern portability: Distinguish the language-dependent differences among Cucumber-family tools from the largely language- and platform-independent pattern catalog.
Why it matters: You can follow the patterns without assuming that every tool implements their solutions in exactly the same way.
Key terms: scenario automation, faster feedback, increased coverage, reduced effort, flakey automation, Cucumber family, Reqnroll, SpecFlow, pattern catalog, language/platform independent
12.1 Automation proof-of-concept
The section explains why a team might temporarily delay scenario automation and how a focused, collaborative proof-of-concept can help it establish an automation solution.
- Temporary automation quality debt: Explain why delaying scenario automation creates quality debt and when taking on that debt briefly can be justified.
Why it matters: A temporary delay is a deliberate trade-off only if the team uses the time well and returns to the unautomated scenarios. - Focused automation proof-of-concept: Describe how the WIMP team uses a proof-of-concept to gain confidence in its automation solution before automating the remaining scenarios.
Why it matters: Trying a design against varied scenarios helps the team build confidence before applying it to every scenario. - Shared proof-of-concept responsibility: Explain why coordinating the POC does not make its automation decisions one person's responsibility.
Why it matters: A coordinator can guide the exploration without being left to solve the automation challenges alone.
Key terms: scenario automation, quality debt, automation-first approach, proof-of-concept (POC), discovery, formulation, automation solution, pairing, ensemble, POC
12.2 Document findings as patterns
This section explains how to capture recurring scenario-automation solutions as design patterns, document and select them by context, and organize them for human and AI use.
- Reusable automation solutions: Explain what a software design pattern is and why reusing a suitable solution to a recurring automation challenge helps keep automation consistent.
Why it matters: Recognizing recurring challenges lets the team reuse approaches that have already worked. - Useful pattern documentation: Describe what a documented automation pattern needs to communicate, regardless of the format used.
Why it matters: Clear documentation makes a successful solution discussable and usable by the rest of the team. - Context-sensitive pattern selection: Justify why a team should check a pattern’s intent, context, constraints, and benefit rather than apply patterns as a shopping list.
Why it matters: Selective use prevents an attractive pattern from making the automation codebase unnecessarily complicated. - Related pattern languages: Explain how pattern languages organize related patterns and what the maps’ directed arcs and labels represent.
Why it matters: Grouping and mapping related solutions helps readers navigate a catalog by the concern they face. - AI-assisted pattern use: Describe how a pattern catalog can guide AI-assisted automation and why its suggestions still need validation.
Why it matters: A catalog can improve generated automation without surrendering judgment about whether a pattern belongs there.
Key terms: software design pattern, recurring challenge, context, descriptive name, challenge, tradeoffs, codebase, intent, constraints, complexity
12.3 Scenario automation tools
Scenario automation tools turn formulated scenarios into executable tests, run them at different scopes, and connect scenario text to automation code supplied by the team.
- Scenario Text and Automation Code: Explain how scenario automation tools use formulated scenarios to verify expected behavior, and distinguish what the tool does from what automation coders must provide.
Why it matters: This distinction keeps the team from expecting the tool to create the automation logic for them. - Executable Scenarios and Run Scope: Describe how ordinary scenarios and scenario outlines become executable tests, and explain what feedback comes from running all, some, or one scenario.
Why it matters: Choosing the run scope gives the team feedback suited to the work at hand. - Platform Dependent Test Execution: Distinguish standalone scenario test runners from tools that integrate with a platform’s native test execution environment.
Why it matters: The way scenarios are launched depends on the tool and platform the team uses.
Key terms: scenarios, scenario automation tools, automation logic, executable test, scenario outlines, example row, subset of scenarios, standalone test execution tool, native test execution environment, platform
12.3.1 Given/When/Then scenario automation tools
This subsection explains that Given/When/Then automation tools share a broad approach but differ in how they register and match scenario steps.
- Step definition registration differences: Distinguish how Cucumber and Reqnroll use Given/When/Then keywords when matching scenario steps to registered expressions.
Why it matters: The same step-definition registration can match differently depending on the tool. - Variation among automation tools: Explain why Given/When/Then scenario automation tools should not be assumed to behave identically, even though they broadly follow the same approach.
Why it matters: Checking the chosen tool’s behavior avoids assuming that examples from another implementation work the same way.
Key terms: step definition registration, registered expression, Gherkin keyword, [StepDefinition] attribute, scenario automation tools, ports, Cucumber-JVM, Reqnroll
12.3.2 Step definitions
This section explains how reusable step definitions turn scenario steps into executable automation, how tools match steps to definitions, and what happens when matching fails.
- Reusable step definition workflow: Explain what a step definition is and how a scenario automation tool uses one for each scenario step to compose an executable test. Describe why definitions can be reused across scenarios rather than writing separate automation logic for every test case.
Why it matters: Reuse means a new scenario often needs automation code for only some of its steps. - Step registration and matching: Trace how a registered step keyword and text expression connect scenario text to a step definition, and distinguish the errors caused by no match and multiple matches.
Why it matters: Matching determines which automation runs and exposes steps that cannot be resolved uniquely.
Key terms: step definition, scenario automation tool, building blocks, suggested step keyword, text expression, undefined step, ambiguous step, method attribute, annotation
12.3.3 Expressions and parameters
This section explains how Cucumber Expressions match scenario steps to step definitions and how parameter values, docstrings, and data tables reach those definitions.
- Expression Matching and Parameters: Explain how a Cucumber Expression matches step text and how placeholders let one step definition handle different test data. Describe how matched values become arguments, including their order when there are multiple placeholders.
Why it matters: It explains how readable steps can share automation code while supplying different values. - Typed and Flexible Expressions: Distinguish quoted {string} values from unquoted, single-word {word} values, and describe the supported ways to accommodate phrasing variations. Explain why that flexibility should not conceal inconsistent business terminology.
Why it matters: Flexible matching is useful only if the scenarios still communicate consistently to readers. - Associated Step Data Arguments: Explain how docstrings and data tables relate to expression matching and how their values are passed to a step definition.
Why it matters: It separates what selects a step definition from the additional data that definition receives. - Data Table Argument Shapes: Describe how a data table with a header row is represented for a step definition, and contrast it with a supported table that has no headers.
Why it matters: The presence of headers changes the shape of the data the automation code receives.
Key terms: Cucumber Expressions, step definition, parameter placeholders, {string}, {word}, optional text, alternate text, unambiguous terms, data tables, docstrings
12.3.4 Custom parameter types
This section explains how custom parameter types convert text captured by Cucumber Expressions into step-definition values, and how Reqnroll registers and extends them.
- Custom parameter type purpose: Explain how a custom parameter type extends Cucumber Expression placeholders by converting captured text into a value for a step definition.
Why it matters: This explains what the placeholder contributes beyond capturing raw text. - Reqnroll transformation registration: Describe the requirements for a Reqnroll transformation method, including its binding class, placeholder name, and regular expression.
Why it matters: These rules determine which placeholder a step can use and what text reaches its converter. - Multiple methods for one type: Explain how Reqnroll can use multiple transformation methods to support different representations of one parameter type.
Why it matters: One parameter type can cover several useful ways of expressing the same kind of value.
Key terms: custom parameter type, Cucumber Expressions, [StepArgumentTransformation], step definition, [Binding], Name, regular expression, capture groups, color parameter type, Color.Red
12.3.5 Matching steps to step definitions
This section explains how step definition expressions are matched to scenario steps, what happens when matching fails or is ambiguous, and why simple expressions are easier to maintain.
- Expanded step definition matching: Explain that optional or alternate text and custom parameter types can expand a step definition expression into multiple possible matches, all of which are checked against step text.
Why it matters: Understanding the expanded possibilities helps explain why a step matches—or unexpectedly matches more than once. - Undefined and ambiguous matches: Distinguish what happens when no step definition matches a step from what happens when more than one matches.
Why it matters: The distinction points you toward either a missing match or competing definitions when execution stops. - Simple expression tradeoff: Justify using multiple simple step definition expressions rather than complex expressions and custom parameter types merely to reduce their number.
Why it matters: Simple expressions can save more debugging time than a smaller number of clever definitions.
Key terms: step text, step definition expression, custom parameter types, regular expressions, undefined step exception, ambiguous step exception, step definition, step definition expressions, custom parameter type definitions, matching logic
12.3.6 Hooks
This section explains how hooks run automation logic around scenario events and how to register, order, and restrict them.
- Hook purpose and timing: Explain that hooks run automation logic not tied to a particular step, and describe when scenario, test-run, and step hooks execute.
Why it matters: Hooks give automation a place for work that does not belong to an individual scenario step. - Annotated hook registration: Describe how most tools register hooks and what the Reqnroll before-scenario example makes possible.
Why it matters: The example shows how a hook can prepare a consistent starting state for each scenario. - Hook ordering and tag restrictions: Explain how multiple hooks for one event can be ordered and how scenario-specific hooks can be limited by tags.
Why it matters: Ordering and tag conditions let a team control when shared automation logic runs.
Key terms: hook, before-scenario hook, after-scenario hook, before-all, after-all, before-step, annotated method, Reqnroll, Order property, tag expression
12.3.7 Organizing glue code
This section explains what glue code is, why it is organized for reuse rather than by feature file, and how glue-class instances keep scenario data isolated.
- Glue Code and Global Availability: Explain what counts as glue code and what its usual global scope means for scenario execution.
Why it matters: Global availability explains why a glue code file’s location does not limit which scenarios can use it. - Reusable Glue Code Organization: Describe how the section recommends grouping step definitions and support code, and justify why step-definition folders should not strictly mirror feature files.
Why it matters: A clear structure makes shared step definitions easier to find and maintain without tying them to one feature. - Scenario Isolated Class Instances: Explain how glue-class instance fields can hold data during a scenario without carrying it into another scenario.
Why it matters: Per-scenario instances let glue code retain data during one scenario without leaking it into another.
Key terms: glue code, step definitions, hooks, custom parameter types, global scope, glue classes, StepDefinitions, Support, Given steps, When and Then steps
12.4 Key principles for “SUITABLE” scenario automation
Introduces SUITABLE, the authors’ eight principles for successful BDD scenario automation, with a brief meaning for each.
- Scenario setup and feedback: Explain how shortcuts, isolated scenarios, and fast feedback contribute to scenario automation.
Why it matters: These principles help automation support BDD’s small, frequent steps without making scenarios depend on one another. - Diagnostics and automation code quality: Describe the principles of understandable diagnostics, test-code quality, and atomic assertions.
Why it matters: These principles make scenario automation easier to understand and maintain. - Business language and loose coupling: Explain what the business-terminology and loose-coupling principles call for.
Why it matters: These principles connect automation to the business domain while avoiding unnecessary coupling.
Key terms: Shortcuts, Isolated scenarios, Evolving with fast feedback, Understandable diagnostics, Test-code quality, Atomic assertions, Business terminology, Loose coupling
12.4.1 Shortcuts
This section explains how to keep scenario setup business-readable while using automation shortcuts to establish the required starting state.
- Intention-revealing context steps: Explain why a scenario should use a brief, named step to describe its starting context rather than list every setup action.
Why it matters: A concise context step communicates the state that matters without obscuring the behavior being specified. - Shortcuts for scenario setup: Explain what an automation shortcut skips and how the team can choose a suitable way to establish a scenario’s required starting state, using the Rebecca example.
Why it matters: Shortcuts let automation reach the state a scenario needs without spending time on unrelated workflows.
Key terms: BRIEF, illustrative examples, named abstraction, Given, shortcut, starting context, step definition, test database, test API
12.4.2 Understandable diagnostics
This subsection explains why scenario automation needs clear diagnostics when new behavior causes failures.
- Understandable failure diagnostics: Explain why scenario automation and the system should provide meaningful error messages and test logs when scenarios fail.
Why it matters: Clear failure information helps the team understand unintended behavior without losing unnecessary time.
Key terms: understandable diagnostics, error messages, test logs, cognitive load
12.4.3 Isolated scenarios
This section explains why automated scenarios need isolated execution contexts so they can run independently, in any order, or in parallel without slowing feedback.
- Independent scenario execution: Explain why each automated scenario must have an execution context unaffected by other scenarios, including the state of external services and resources when scenarios run in parallel.
Why it matters: Isolation keeps feedback useful and timely as the scenario suite grows.
Key terms: isolated scenarios, execution context, feedback loop, external services and resources
12.4.4 Test-code quality
This section explains why test automation code deserves the same attention to quality as production code.
- Test Automation Code Quality: Justify treating test automation code as at least as important as production code by describing its roles beyond guiding implementation.
Why it matters: Automation remains useful after a scenario first passes, so its code needs to be maintained with care.
Key terms: test automation code, business-readable specification, test-code quality, production code
12.4.5 Atomic assertion
This section explains why multiple assertions in one step definition can hide diagnostic information and recommends checking and reporting multiple verifications through a single atomic assertion.
- Atomic assertion principle: Explain why sequential assertions can leave a failure incompletely diagnosed and how a single atomic assertion can report multiple verification results together.
Why it matters: A failure is quicker to diagnose when its report includes the values needed to investigate all the related checks.
Key terms: assertions, step definition, atomic assertion, diagnostic message
12.4.6 Business terminology
This section recommends using business-domain terminology consistently throughout a system to make its behavior and design easier to reason about.
- Consistent business domain terminology: Explain why named abstractions help people reason about complex systems and why the same business-domain terms should be used from high-level specifications to low-level diagnostic output.
Why it matters: Consistent names help people connect a specification with the system that implements it.
Key terms: abstractions, business domain terminology, Ubiquitous Language
12.4.7 Loose coupling
Loose coupling lets scenario automation choose how to exercise behavior and replace dependencies, improving feedback speed and test coverage.
- Loose coupling for automation: Explain how well-defined interfaces let automation coders choose where to initiate behavior and substitute dependencies, and why those choices help them verify behavior efficiently and thoroughly.
Why it matters: These choices help verify more behavior without significantly slowing feedback.
Key terms: loose coupling, well-defined interfaces, dependencies, speed of feedback
12.4.8 Evolving with fast feedback
This section explains how BDD advances in small scenario-driven steps and why fast feedback makes those steps practical.
- Scenario Driven Development Cycle: Trace the cycle of specifying unimplemented behavior in a scenario, automating it and seeing it fail, then implementing enough functionality to make it pass without breaking existing scenarios.
Why it matters: The cycle shows how each new scenario guides a small change while existing scenarios guard against regressions. - Fast Feedback and Diagnosis: Explain why small BDD steps need fast, frequent feedback from the build process and stakeholders, and how a short test suite enables full-suite runs on every commit and helps locate failures.
Why it matters: Quick, frequent results let the team stay focused and make a newly introduced failure easier to investigate.
Key terms: iterative and incremental, scenario, automated scenarios, fast, frequent feedback, build process, stakeholders, full test suite, every commit
12.5 Automation strategies
This section explains how to choose a context-dependent scenario automation strategy that balances fast feedback with confidence in the application’s behavior.
- BDD and automation strategy: Explain why BDD does not prescribe how scenarios must be automated. Describe what scenario automation tools provide and what the team must still decide.
Why it matters: The team can choose an approach that fits the application rather than assuming every scenario needs a browser. - Application interface trade-offs: Distinguish inner, outer technical, and UI interfaces for scenario automation, and explain the trade-off between fast inner-interface tests and coverage of the external context.
Why it matters: Choosing where to exercise behavior affects both feedback speed and the chance of catching integration problems. - Real infrastructure versus stubs: Explain how to choose between real external infrastructure and stubs for scenario automation, and what additional checks may be needed when stubs are used.
Why it matters: Stubs can speed up feedback, but the testing strategy must still provide confidence that real integrations work. - Test Automation Pyramid intent: Explain the Test Automation Pyramid’s intended balance of tests and how skipping costly layers or dependencies can help achieve it.
Why it matters: The pyramid helps explain why confidence does not require every test to traverse every layer. - Documented automation strategy: Describe how AI can help document and check an agreed automation strategy, and why its output should be kept concise.
Why it matters: A short, maintained record makes the team’s strategy easier to follow and revisit.
Key terms: BDD, automation strategy, scenario automation tools, business-readable specification steps, inner, outer, outer technical, UI, integration error, external infrastructure
12.6 Summary
Where this is used later
- (chapter intro) (chapter 19) · Assumes the role of automated BDD scenarios in providing feedback.
- Local Scenario Context pattern: learning to remember (chapter 13) · Frames the first pattern as a generally applicable solution rather than one limited to the book’s example language.
- Test Driver pattern: separating automation logic (chapter 15) · Supports deciding whether a particular automation pattern fits a layering challenge.
- (chapter intro) (chapter 13) · Introduces one of the grouped pattern languages for scenario automation.
- Terminology Promotion pattern: expressing domain-specific data (chapter 14) · Cucumber expression placeholders and step definition parameters
- Data Table Accessor pattern: handling data table parameters (chapter 14) · Data table accessors work with table arguments passed to automation code.
- Introduce and Refer pattern: separating introduction and reference (chapter 13) · Custom parameter types used in scenario automation.
- What about “they”? (chapter 13) · Custom parameter types in scenario automation.
Concepts explained in this chapter
- parameter typeA user-defined parameter type that matches captured text and converts it into a value for a step definition.
- design patternA reusable description of a solution to a recurring software design problem.
- external infrastructureProduction dependencies outside an application, such as databases, external services, file systems, or system time.
- stubA test double that substitutes for a real dependency and provides predetermined behavior or responses.
- test strategyA planned approach defining the testing activities, scope, and coverage needed to evaluate a software product.
- A Pattern LanguageA structured collection of interrelated patterns that helps identify an appropriate pattern for a specific context.
- step definitionAutomation code associated with a behavior-specification step that performs or checks the step’s behavior.
- SUITABLEA mnemonic organizing eight principles for designing effective scenario automation.
- test isolationThe principle that each scenario should run independently, without interference from other scenarios or shared state.
- cucumber expressionA Cucumber pattern syntax for matching Gherkin step text to a step definition and extracting parameter values.
- StepArgumentTransformation attributeA Reqnroll attribute that registers a method to transform captured text into a value for a step definition.
- ubiquitous languageA shared vocabulary used by domain experts and software practitioners to describe business concepts and system behavior consistently.
- scenario automation toolSoftware that supports automating scenarios by connecting human-readable behavior descriptions to executable automation.
- execution contextScenario-scoped state and information available to steps during one scenario execution.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
