Chapter 21: Stagecraft patterns for interacting with external resources
This introduction explains why costly external resources can slow scenario tests and why sharing them complicates parallel execution.
After this chapter you will be able to
- External Resource Costs and Isolation. Explain how external resources can slow test feedback and why difficulty creating separate instances can force scenarios to share resources, creating isolation issues.
- Parallel Execution Tradeoff. Explain why running scenarios in parallel can improve feedback time and why shared resources can make parallel execution unsafe.
- Run-Level Resource Initialization. Explain why shared external resources can be initialized with a before-test-run hook rather than checked before every scenario, using the database schema example.
- Exclusive resource leases. Explain how a scenario acquires exclusive use of a pooled external resource and returns it when finished, and why this helps when resources are costly or impossible to create ad hoc.
- Parallel pool capacity. Explain how matching the maximum pool size to the number of test workers supports parallel scenarios, and what the WIMP implementation does if an additional request cannot be satisfied.
- Lazy thread-safe pooling. Describe when the WIMP pool creates a DatabaseContext and why its acquire-and-release infrastructure must be thread-safe.
- Dependency injection integration. Describe how WIMP switches database access from a fixed factory to pooled DatabaseContext instances through dependency injection, including how a scenario returns its database.
- Centralized test file paths. Explain how the Test File System pattern puts knowledge of test-file locations in one helper class and why this makes changes easier.
- Scenario step timing mismatch. Explain how a sequential scenario can check an asynchronous resource at the wrong time: a Then step may run before the requested work finishes, or after a change it expected to detect has already occurred.
- Limits of asynchronous keywords. Distinguish nonblocking asynchronous I/O from detecting completion of work started by an external resource. Explain why awaiting an HTTP request is insufficient if the resource produces its result later in a background process.
- Active Waiting Loop. Explain how Active Waiting checks for an asynchronous result when its arrival time is unknown.
- Timeout Failure Diagnosis. Explain how to bound the waiting loop and distinguish a timeout from a negative operation result.
- Completion Events Instead of Polling. Explain that the Completed Event pattern lets automation wait for a notification from an asynchronous resource rather than repeatedly query for a result, provided the resource can signal completion.
- Synchronous processing in tests. Explain when to use the Synchronous Testing pattern and how it lets a scenario check a result without waiting for asynchronous processing.
What it builds on
- Scenario Resource pattern: managing test resources (chapter 16) · Managing the lifetime of resources needed by a scenario.
- Challenges of using real databases for test execution (chapter 18) · Challenges of testing with shared external resources.
- Clean Start pattern: start each scenario with an empty database (chapter 18) · The earlier Clean Start database reset used a before-scenario hook.
- Shareable Scenario Context pattern: sharing data between step definitions in different classes (chapter 13) · Shareable Scenario Context relies on objects available within a scenario context.
- Skippable Reset pattern: do not reset database after read-only scenario executions (chapter 18) · The Skippable Reset example that tracks whether the menu table was modified
- Stub Dependency pattern: stubbing an external service (chapter 15) · The WIMP stub time service is an example of the Stub Dependency pattern.
Section by section
Introduction
This introduction explains why costly external resources can slow scenario tests and why sharing them complicates parallel execution.
- External Resource Costs and Isolation: Explain how external resources can slow test feedback and why difficulty creating separate instances can force scenarios to share resources, creating isolation issues.
Why it matters: These costs explain why external resources need special consideration in a test automation strategy. - Parallel Execution Tradeoff: Explain why running scenarios in parallel can improve feedback time and why shared resources can make parallel execution unsafe.
Why it matters: The speed benefit of parallel tests has to be weighed against interference between scenarios.
Key terms: external resources, fast feedback, test isolation, parallel execution, shared resources
21.1 Test Environment Initialization pattern: getting ready for test execution
This section explains how to initialize external resources efficiently for a test run, clean them up afterward, and account for the limits of test-run hooks.
- Run-Level Resource Initialization: Explain why shared external resources can be initialized with a before-test-run hook rather than checked before every scenario, using the database schema example.
Why it matters: A once-per-run check can keep shared resources ready without repeating the work for every scenario. - Test-Run Hook Context Limits: Explain how running outside a scenario context limits what before-test-run and after-test-run hooks can access.
Why it matters: Code moved into a test-run hook cannot assume the same objects are available as in a scenario. - Post-Run Cleanup Reliability: Describe what an after-test-run hook can clean up and why its errors and output need care.
Why it matters: Cleanup is useful after failures, but a team should not assume its output or execution is always visible and reliable. - On-Demand Initialization Alternative: Distinguish on-demand initialization from a before-test-run hook and explain why it cannot replace end-of-run cleanup.
Why it matters: Initialization has an alternative trigger, but cleanup still needs an end-of-run mechanism.
Key terms: Test Environment Initialization pattern, before-test-run hook, before-scenario hook, database schema, scenario context, dependency injection (DI), static methods, database context factory, after-test-run hook, clean-up activities
21.2 Resource Pooling pattern: leasing pooled resources to scenarios running in parallel
This section explains how leasing costly external resources from a pool lets scenarios run in parallel without creating a new resource for every scenario.
- Exclusive resource leases: Explain how a scenario acquires exclusive use of a pooled external resource and returns it when finished, and why this helps when resources are costly or impossible to create ad hoc.
Why it matters: Leasing lets scenarios reuse expensive resources without using the same instance simultaneously. - Parallel pool capacity: Explain how matching the maximum pool size to the number of test workers supports parallel scenarios, and what the WIMP implementation does if an additional request cannot be satisfied.
Why it matters: Pool capacity determines whether parallel workers can all obtain the resources they need. - Lazy thread-safe pooling: Describe when the WIMP pool creates a DatabaseContext and why its acquire-and-release infrastructure must be thread-safe.
Why it matters: Lazy creation avoids preparing unused databases, while thread safety protects access from concurrent scenarios. - Dependency injection integration: Describe how WIMP switches database access from a fixed factory to pooled DatabaseContext instances through dependency injection, including how a scenario returns its database.
Why it matters: DI confines the pooling change to configuration rather than spreading it through database-using glue code. - Per-database state tracking: Explain why information about a pooled database, such as whether its menu table was modified, belongs on that database’s DatabaseContext rather than in one static field.
Why it matters: Keeping modification state with its database prevents one pooled database’s state from standing in for another’s.
Key terms: Resource Pooling pattern, resource pool, leased, test workers, parallel execution, DatabasePoolDriver, AcquireDatabase, ReleaseDatabase, lazy initialization, thread-safe
21.3 Test File System pattern: handling test files in a uniform way
This section explains how the Test File System pattern centralizes file-path decisions for test automation across execution environments.
- Application and automation file systems: Distinguish the file system used by the application under test from the one used by test automation, and explain why the section recommends stubbing the former during testing.
Why it matters: The distinction keeps the recommendation to stub application file operations separate from the pattern for managing test files. - Environment dependent file locations: Explain why test automation should not assume that the current folder or familiar input and temporary-file locations will work in every execution environment.
Why it matters: A file operation that works locally may write to the wrong place or fail when tests run elsewhere. - Centralized test file paths: Explain how the Test File System pattern puts knowledge of test-file locations in one helper class and why this makes changes easier.
Why it matters: Centralizing path decisions lets the automation adapt to a new execution environment without hunting through scattered file operations. - Helper paths and file names: Describe the kinds of paths and file names the example TestFileSystem class supplies, and how automation code obtains the helper.
Why it matters: These services give file-using automation a consistent way to obtain usable locations and names.
Key terms: SUT, stub file system, test automation solution, current folder, test input files, TEMP environment variable, Test File System pattern, TestFileSystem, test input, test output
21.4 Patterns for asynchronous resources
This section explains why asynchronous external resources create timing problems for sequential scenarios and why language-level async constructs alone may not solve them.
- Scenario step timing mismatch: Explain how a sequential scenario can check an asynchronous resource at the wrong time: a Then step may run before the requested work finishes, or after a change it expected to detect has already occurred.
Why it matters: It explains why a scenario can fail to observe the expected result even when the resource is working. - Limits of asynchronous keywords: Distinguish nonblocking asynchronous I/O from detecting completion of work started by an external resource. Explain why awaiting an HTTP request is insufficient if the resource produces its result later in a background process.
Why it matters: It prevents confusing completion of a request with completion of the work that request started.
Key terms: asynchronous resources, scenario execution, Then step, asynchronous programming constructs, asynchronous I/O, await, background process
21.4.1 Active Waiting pattern: keep polling the resource
This section explains how to poll for an asynchronous result in scenario automation, including the timing, race-condition, and timeout problems that polling introduces.
- Active Waiting Loop: Explain how Active Waiting checks for an asynchronous result when its arrival time is unknown.
Why it matters: Polling lets a scenario wait for an unknown completion time without assuming the result is already available. - Polling Delay Tradeoffs: Explain why a fixed sleep is discouraged and how the delay between polls affects test execution.
Why it matters: Choosing a polling interval means balancing repeated-query cost against time wasted waiting to notice a completed operation. - Early Completion Race: Describe how to detect a change when an asynchronous operation finishes before the waiting loop begins.
Why it matters: A baseline keeps a fast operation from escaping a check that starts only after the operation has finished. - Timeout Failure Diagnosis: Explain how to bound the waiting loop and distinguish a timeout from a negative operation result.
Why it matters: A bounded wait and an accurate failure message prevent hangs and make the cause of a failed scenario clear.
Key terms: Active Waiting, polling, asynchronous operation, fixed wait, sleep, delay between two subsequent polls, When step, Then step, system status, timeout period
21.4.2 Completed Event pattern: getting a notification when the resource produced some result
The section explains how completion notifications can replace polling for asynchronous results, why a timeout remains necessary, and how to provide notifications when a resource lacks them.
- Completion Events Instead of Polling: Explain that the Completed Event pattern lets automation wait for a notification from an asynchronous resource rather than repeatedly query for a result, provided the resource can signal completion.
Why it matters: It explains both the efficiency gain and the capability the resource must provide. - Overall Completion Timeout: Explain why event-based waiting still needs an overall timeout.
Why it matters: A timeout prevents automation from waiting indefinitely for an event that never arrives. - Providing Completion Notifications: Describe the section’s recommendations for obtaining completion events, including a façade when a repeatedly used resource cannot provide them directly.
Why it matters: A façade can give multiple callers a simpler event-based interface even when the underlying resource lacks one.
Key terms: Completed Event pattern, Active Waiting pattern, notification (event), overall timeout, notification, completion event functionality, façade, anti-corruption layer, active waiting
21.4.3 Synchronous Testing pattern: turning the asynchronous behavior to synchronous for testing
This section explains when synchronous processing can simplify tests that depend on asynchronous resources, and what those tests may fail to detect.
- Synchronous processing in tests: Explain when to use the Synchronous Testing pattern and how it lets a scenario check a result without waiting for asynchronous processing.
Why it matters: It can make a scenario's result straightforward to check when timing is not the behavior under test. - Limits of synchronous testing: Explain why replacing asynchronous processing with synchronous processing can leave important defects undetected.
Why it matters: A simpler test is useful only if its blind spots are acceptable for the automation goal.
Key terms: Synchronous Testing pattern, asynchronous resource, synchronous processing, thread-safety issues, automation goal, main thread
21.5 Summary
Concepts explained in this chapter
- resource poolingA pattern that assigns reusable instances of a shared resource for exclusive use while work executes.
- test file systemA pattern that centralizes file paths and abstracts file-system interactions for test input, output, and temporary files.
- test environment initializationA pattern for efficiently initializing external resources shared by tests and cleaning them up when needed.
- busy-waiting loopA pattern for handling an asynchronous result by repeatedly checking for completion until the result appears or a timeout occurs.
- synchronous testingA testing pattern that runs an asynchronous operation synchronously when asynchrony is not the behavior under test.
- completed event patternA pattern for waiting for an asynchronous operation to finish by receiving a completion event instead of repeatedly checking its status.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
