Chapter 20: Staging patterns for adapting test execution
The chapter introduction explains why scenario execution must accommodate the different external services and customized deployments a system supports.
After this chapter you will be able to
- Deployment configuration variety. Explain why testing a system requires considering more than a choice between stub and production-like external resources. Describe how supported services and customized deployments create multiple configurations to test.
- Environment Settings and Test Parameters. Distinguish settings that select the systems or resources a scenario runs against from parameters that specify how it runs. Explain why environments need different settings and why verification may use several parameter values.
- Configuration Sources and Overrides. Describe the configuration sources named in the section and explain why a BDD automation solution might combine them.
- Configuration loading encapsulation. Explain why the WIMP team puts scenario execution settings behind a TestConfigurationProvider rather than exposing configuration management throughout the automation code.
- Configuration Driven Scenario Coverage. Explain how the Parameterized Execution pattern extends coverage by rerunning BDD scenarios with different automation configurations, while preserving fast and robust feedback as the first priority.
- Variation Costs And Relevance. Justify when parameterized execution is worthwhile by weighing the risks it addresses against time and infrastructure costs, and by identifying scenarios affected by and compatible with each parameter value.
- Configuration-dependent specifications. Explain why verifying a configurable application requires attention to scenario wording as well as test execution. Describe how a detail such as currency can make an otherwise reusable scenario inappropriate for another deployment.
- Configuration markers and resolution. Describe how the Parameterized Scenario pattern marks configuration-dependent parts of scenarios and resolves them for test actions. Give an example involving currency or terminology.
- Configuration-sensitive expected outcomes. Explain why parameterizing scenario text may require redesigning a scenario's expected outcomes, especially for sales tax.
What it builds on
- Stub Dependency pattern: stubbing an external service (chapter 15) · Provides the earlier context of using a stub for an external dependency.
- Polymorphic Driver pattern: reuse driver interfaces (chapter 15) · The Polymorphic Driver pattern is cited as an earlier use of test parameters.
- Automation strategies (chapter 12) · Assumes the idea of choosing an automation strategy.
- Scenario outlines and examples tables (chapter 8) · Provides the scenario outlines mentioned as an alternative when few scenarios benefit.
- Custom parameter types (chapter 12) · Custom parameter types in scenario automation
Section by section
Introduction
The chapter introduction explains why scenario execution must accommodate the different external services and customized deployments a system supports.
- Deployment configuration variety: Explain why testing a system requires considering more than a choice between stub and production-like external resources. Describe how supported services and customized deployments create multiple configurations to test.
Why it matters: Recognizing the range of supported configurations helps the team see why one test setup may not provide enough coverage.
Key terms: configuration, external services, customized deployments, localized, white-labelled
20.1 Test Configuration
This section distinguishes environment configuration from test parameters and explains how automated scenarios can use defaults and overrides across execution contexts.
- Environment Settings and Test Parameters: Distinguish settings that select the systems or resources a scenario runs against from parameters that specify how it runs. Explain why environments need different settings and why verification may use several parameter values.
Why it matters: This distinction helps you recognize whether a setting adapts a scenario to its surroundings or varies the way it is tested. - Local Defaults and Overrides: Explain why configuration usually has defaults for local execution and why those values may be overridden for CI/CD or special verification.
Why it matters: Defaults keep routine local testing easy without fixing every run to local settings. - Configuration Sources and Overrides: Describe the configuration sources named in the section and explain why a BDD automation solution might combine them.
Why it matters: Multiple sources let a team retain ordinary settings while adapting a run to its environment.
Key terms: environment configuration, test parameters, default values, local execution, CI/CD pipeline, configuration files, environment variables, console arguments
20.2 Test Configuration Provider pattern: provide uniform access to configuration settings
The section shows how a Test Configuration Provider hides configuration-loading details while allowing scenario execution settings to vary between local and CI/CD runs.
- Configuration loading encapsulation: Explain why the WIMP team puts scenario execution settings behind a TestConfigurationProvider rather than exposing configuration management throughout the automation code.
Why it matters: Automation code can use settings without each part having to manage how those settings are loaded. - File defaults and overrides: Describe how the team's file-based settings support defaults and how environment variables customize execution on CI/CD without changing the file.
Why it matters: The same automation can run locally and in CI/CD with different settings without editing its configuration file. - Flat versus sectioned settings: Distinguish the provider's flat properties from section-objects and explain when the sectioned design becomes preferable.
Why it matters: Grouping settings by section keeps a growing provider manageable without changing its basic role.
Key terms: Test Configuration Provider pattern, TestConfigurationProvider, configuration settings, configuration file, environment variables, CI/CD pipeline, UseStub, flattened, section-objects, DatabaseTestConfiguration
20.3 Parameterized Execution pattern: extend the coverage of BDD scenarios
This section explains how running BDD scenarios with different test configurations can extend coverage, and how to choose and manage those variations.
- Configuration Driven Scenario Coverage: Explain how the Parameterized Execution pattern extends coverage by rerunning BDD scenarios with different automation configurations, while preserving fast and robust feedback as the first priority.
Why it matters: You can check more application situations without writing a separate scenario for every automation choice. - Variation Costs And Relevance: Justify when parameterized execution is worthwhile by weighing the risks it addresses against time and infrastructure costs, and by identifying scenarios affected by and compatible with each parameter value.
Why it matters: Selecting useful combinations avoids paying to rerun scenarios that gain little or cannot run under a given configuration. - Selective Runs And Skips: Describe how tag filtering can limit parameterized runs when most but not all scenarios benefit, and explain the alternative of dynamically skipping unsupported combinations.
Why it matters: Filtering or skipping keeps unsuitable scenario–configuration combinations out of execution. - Repeatable Variation Across Runs: Explain why predictable variation can be preferable to random choices during debugging, and describe how a CI/CD pipeline can run different parameter sets.
Why it matters: Regularly checking alternatives is more useful when a failing run can be reproduced and investigated.
Key terms: Parameterized Execution pattern, test configuration parameter, fast and robust feedback, parameterized execution, scenario outline examples, test parameters, tag filtering, skipping, execution report, CI/CD pipeline
20.4 Parameterized Scenario pattern: specify and verify requirement variations
This section explains how Parameterized Scenarios adapt specifications and their automation to verify different deployment configurations.
- Configuration-dependent specifications: Explain why verifying a configurable application requires attention to scenario wording as well as test execution. Describe how a detail such as currency can make an otherwise reusable scenario inappropriate for another deployment.
Why it matters: A scenario should remain meaningful for every deployment it is meant to verify. - Configuration markers and resolution: Describe how the Parameterized Scenario pattern marks configuration-dependent parts of scenarios and resolves them for test actions. Give an example involving currency or terminology.
Why it matters: Marking variable wording lets one set of scenarios describe and validate multiple configured deployments. - Configuration-sensitive expected outcomes: Explain why parameterizing scenario text may require redesigning a scenario's expected outcomes, especially for sales tax.
Why it matters: A scenario cannot correctly verify several configurations if it assumes the same expected result for all of them. - Alternative value substitutions: Describe the alternatives the section gives to explicit placeholders for supplying configuration-specific scenario text.
Why it matters: The notation and substitution method can vary while serving the same configuration-dependent specification.
Key terms: software factory, deployment configurations, test configurations, Parameterized Scenario pattern, special notation, custom parameter types, configuration settings, sales tax behaviors, Then step definitions, expected value
20.5 Summary
Concepts explained in this chapter
- parameterized executionA pattern in which automation code uses test parameters to vary execution choices and extend scenario coverage.
- parameterized scenario patternA scenario pattern that uses configurable values or notation to specify and verify behavior across different deployment configurations.
- test configuration provider patternA pattern that centralizes how test configuration settings are loaded and made available.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
