Chapter 11: Chapter 11. Strategies for implementing acceptance tests
The introduction frames acceptance-test implementation as a choice between testing the whole system and testing at lower levels, considering cost versus value.
After this chapter you will be able to
- Testing level tradeoffs. Explain why whole-system testing is valuable and why testing below that level can also be worthwhile.
- Essential acceptance test scope. Explain how to choose what an automated acceptance test should verify for a user story.
- Human and automated testing. Distinguish the roles of manual testing and automated regression tests.
- Story focused test scope. Explain how a story guides the focus of its acceptance tests, using calculation or card-validation behavior as an example.
- UI behavior test choice. Explain when an acceptance test should use the actual UI, using the session-timeout redirect story as an example.
- Interface turbulence and maintenance. Explain how the interface chosen for acceptance tests affects maintenance, using the credit-card validation example. Describe the additional consideration of how easy tests are to create.
- Story focused test expression. Explain why acceptance tests should be expressed in story and domain terms rather than cluttered with web-page navigation details, even when they operate through the UI.
- Technology Limits and Entry Points. Explain how a technology platform’s hard limits and available entry points constrain acceptance-testing options.
- Technology Biases and Tradeoffs. Explain how a technology stack can favor some feasible testing approaches over others, using an example from the section.
- Architecture Creates Testing Options. Explain how a testable architecture can provide an alternative to testing through the UI.
- End-to-end black-box testing. Explain how end-to-end acceptance tests exercise a system and why this approach is valuable to stakeholders.
- UI brittleness and effort. Explain why end-to-end tests that use the UI can be brittle and laborious, and how the section suggests reducing these risks.
- Speed and certainty trade-off. Explain how slow end-to-end tests affect repeated testing, full-suite feedback, and development pace.
- Testing beneath the UI. Explain how crawling under the skin connects acceptance tests to the system beneath its concrete UI and reduces maintenance from UI changes.
- Simpler test implementation. Explain why a programming interface can make acceptance tests easier and more robust to implement than a concrete UI, using the product-selection example.
- Missing UI test coverage. Explain the testing trade-off when acceptance tests bypass the UI and how the section proposes balancing it.
- Direct business logic testing. Describe how internals-oriented acceptance tests exercise specific business or application logic without going through a UI or external API, using validation rules as an example.
- Precision and specification clarity. Explain why isolating business rules in acceptance tests can help the team and customer find specification problems.
- Acceptance and unit test language. Distinguish internals-oriented acceptance tests from unit tests in terms of their audiences and vocabulary, even when they exercise similarly small pieces of code.
- Limits of selective testing. Explain why internals-oriented acceptance tests supplement rather than replace broader acceptance tests, and where they are especially useful.
- Stubbing for testability. Explain how stubbing out the irrelevant replaces components in acceptance tests and why teams do it, using an example.
- Supplementary integration testing. Explain why stubbed acceptance tests need to be supplemented with tests using real components, and describe the balance recommended here.
- Testing backdoor access. Explain what a testing backdoor is and how acceptance tests can use a database as one.
- Technology and connection strategy. Explain why the technology used by a system matters when choosing how to connect acceptance tests to it.
- Library Acceptance Test Interface. Explain why testing a library through its public programming interface can be considered both going under the skin and testing end to end, and identify the remaining interface choice.
- Technical Test Expression Choices. Explain why library stories and acceptance tests can be technical, and describe the choice between Java-based tests and a more accessible format.
- Hybrid distributed testing strategy. Explain why a distributed system might use mostly under-the-skin acceptance tests alongside a smaller set of end-to-end tests.
- Machine interface automation. Explain why end-to-end acceptance testing can be practical for faceless distributed systems.
- End-to-end CLI testing. Explain why testing a console application end-to-end through its UI is often practical and preferred.
- Alternative CLI testing strategies. Explain when testing below the console UI or using stubs may be useful, and what bypassing the UI leaves to test separately.
- GUI testing complexity. Explain why desktop GUI acceptance tests are difficult to automate, including the challenges of timing and accessing components.
- Remote application testing hooks. Explain how exposed hooks give separate tests access beneath the UI, including their abstraction and cost tradeoffs.
- Same process test arrangements. Distinguish launching the application from tests from embedding tests in the application, and explain their practical tradeoffs.
- Web UI testing tradeoffs. Explain the tradeoffs between testing a web application through its UI and testing beneath the web layer.
- Browser simulation capabilities. Describe how browser simulation libraries interact with a web application and why their convenience may not be sufficient for every application.
- Real browser control tradeoffs. Explain what is needed to control a real browser in acceptance tests and the practical costs of doing so.
- Acceptance test design complexity. Explain why acceptance tests for complex systems need design and refactoring, like production code and unit tests.
- Slow test feedback. Explain how long acceptance test execution delays feedback about changes that break other parts of a system.
- Test grouping for setup. Explain how grouping acceptance tests can reduce setup and restoration work.
- Distributed test execution patterns. Describe the three distributed execution patterns and explain why scaling target systems matters.
- Dependent test setup trade-offs. Explain how shared setup can speed up tests and why dependencies make the suite harder to maintain and interpret.
- Transaction rollback for restoration. Explain how transaction rollback speeds database restoration and state its feasibility conditions.
- Soft boot state reset. Describe a soft boot and explain its benefit and limitation.
- Clear acceptance test structure. Explain how ordinary refactoring techniques reduce acceptance-test complexity and make tests communicate in the language of the user story.
- Reusable setup context objects. Describe how context objects simplify common acceptance-test setups and hide setup hierarchies from test cases.
- Caching context state snapshots. Explain how a caching context object speeds up repeated setup and what support and limitation the section identifies.
- Shared test data set. Explain why acceptance tests should share one data set and how that set can still be partitioned into manageable assets.
- Generated test data. Explain why generating a data set can make it easier to manage, and describe how the target system's persistence implementation can help.
- Diffable data set definitions. Explain why and how to version-control a test data set definition in a usable format.
What it builds on
- Format of a story (chapter 9) · User stories as a way to express requirements.
- Properties of acceptance tests (chapter 9) · Acceptance tests as checks on requirements.
- Guidelines for testable designs (chapter 4) · Design choices can make software easier to test.
- Should we test against the UI? (chapter 9) · The choice of whether acceptance tests should exercise the UI.
- Fixtures: combinations of tables and classes (chapter 10) · Fixtures as the connection between acceptance-test tables and code.
- Tests as a shared language (chapter 1) · The role of tests as a shared language
- Test documents contain fixture tables (chapter 10) · Test documents with fixture tables
- Implementing acceptance tests (chapter 9) · Implementing acceptance tests
Section by section
Introduction
The introduction frames acceptance-test implementation as a choice between testing the whole system and testing at lower levels, considering cost versus value.
- Testing level tradeoffs: Explain why whole-system testing is valuable and why testing below that level can also be worthwhile.
Why it matters: This tradeoff helps you consider where acceptance tests should interact with the system.
Key terms: end to end, whole system, cost versus value
11.1 What should acceptance tests test?
This section explains how to focus automated acceptance tests on essential story requirements while combining automation with human testing.
- Essential acceptance test scope: Explain how to choose what an automated acceptance test should verify for a user story.
Why it matters: Focusing on the story’s essential requirements helps avoid treating exhaustive end-to-end automation as the goal. - Human and automated testing: Distinguish the roles of manual testing and automated regression tests.
Why it matters: Automation and human judgment complement each other rather than making either kind of testing unnecessary.
Key terms: acceptance tests, user stories, end-to-end, manual, visual testing, automated regression tests, risks
11.1.1 Focus on what’s essential
This section explains how to focus acceptance tests on the behavior a story requires, using the UI when it is essential to that behavior.
- Story focused test scope: Explain how a story guides the focus of its acceptance tests, using calculation or card-validation behavior as an example.
Why it matters: Focusing on the required behavior avoids making tests depend on unrelated presentation details. - UI behavior test choice: Explain when an acceptance test should use the actual UI, using the session-timeout redirect story as an example.
Why it matters: The UI belongs in the test when the user-facing behavior is what the story is about.
Key terms: acceptance tests, domain logic, presentation, user story, actual UI, web interface, end-to-end scenario, session
11.1.2 Avoid turbulent interfaces
This section explains how to choose acceptance-test interfaces that limit maintenance from unrelated changes and keep tests focused on the story.
- Interface turbulence and maintenance: Explain how the interface chosen for acceptance tests affects maintenance, using the credit-card validation example. Describe the additional consideration of how easy tests are to create.
Why it matters: Choosing where tests connect can prevent unrelated interface changes from causing widespread rework. - Story focused test expression: Explain why acceptance tests should be expressed in story and domain terms rather than cluttered with web-page navigation details, even when they operate through the UI.
Why it matters: Story-focused tests make the behavior being checked easier to see without navigation details obscuring it.
Key terms: turbulence, interface, system under test, cost of change, story, domain, user interface, navigation
11.1.3 Cross the fence where it is lowest
This section explains how technology constrains and favors acceptance-testing approaches, and how architecture can create additional testing options.
- Technology Limits and Entry Points: Explain how a technology platform’s hard limits and available entry points constrain acceptance-testing options.
Why it matters: Knowing the hard limits helps you choose an approach the technology can actually support. - Technology Biases and Tradeoffs: Explain how a technology stack can favor some feasible testing approaches over others, using an example from the section.
Why it matters: A feasible approach may still be costly, so its benefits should justify the effort. - Architecture Creates Testing Options: Explain how a testable architecture can provide an alternative to testing through the UI.
Why it matters: Design choices can expand your testing options rather than leaving you confined to the existing interface.
Key terms: hard limits, entry points, system under test, technology stack, white box, UI components, API-level access, testable, software architecture, service layer
11.2 Implementation approaches
This section introduces the acceptance-testing approaches discussed in the sections that follow.
11.2.1 Going end-to-end
This section explains end-to-end acceptance testing, its real-world coverage and stakeholder benefits, and the costs of UI dependence and slow feedback.
- End-to-end black-box testing: Explain how end-to-end acceptance tests exercise a system and why this approach is valuable to stakeholders.
Why it matters: Testing through the real interface makes acceptance tests both representative and easier for stakeholders to understand. - Fixture translation for web tests: Describe the role of a web acceptance-test fixture in connecting customer language to the application.
Why it matters: Fixtures bridge readable acceptance tests and the steps needed to operate a web application. - UI brittleness and effort: Explain why end-to-end tests that use the UI can be brittle and laborious, and how the section suggests reducing these risks.
Why it matters: Recognizing UI-driven brittleness and effort helps keep acceptance tests useful and maintainable. - Speed and certainty trade-off: Explain how slow end-to-end tests affect repeated testing, full-suite feedback, and development pace.
Why it matters: Fast feedback lets the team catch failures while the relevant work is still fresh.
Key terms: end-to-end, real interface, black box, fixture classes, WebFixture, jWebUnit, customer’s language, brittle, UI, business logic
11.2.2 Crawling under the skin
This section explains how acceptance tests can operate beneath the concrete UI, with examples, benefits, and trade-offs.
- Testing beneath the UI: Explain how crawling under the skin connects acceptance tests to the system beneath its concrete UI and reduces maintenance from UI changes.
Why it matters: This strategy helps keep acceptance tests useful when the interface changes. - Architecture dependent access channels: Describe the weblog examples of accessing a service layer or an abstract UI, and how the system's architecture enables the choice.
Why it matters: The available test entry points depend on how the application is structured and deployed. - Simpler test implementation: Explain why a programming interface can make acceptance tests easier and more robust to implement than a concrete UI, using the product-selection example.
Why it matters: A suitable API can avoid fragile UI element selection and simplify automation. - Missing UI test coverage: Explain the testing trade-off when acceptance tests bypass the UI and how the section proposes balancing it.
Why it matters: Faster, less fragile tests do not eliminate the need to assess the actual interface. - Stakeholder accessible test language: Explain how a shared domain-specific language can keep acceptance tests understandable to nontechnical stakeholders when their implementation bypasses the UI.
Why it matters: Tests can remain a shared communication tool even when their implementation is technical. - Test interface design tension: Describe the tension between exposing functionality for a test interface and encapsulation, along with the possible design benefit.
Why it matters: A test interface changes the system's design as well as how tests access it.
Key terms: crawling under the skin, abstract UI, application and domain logic, internal service layer API, JNDI, JVM, abstract UI API, event queue, programming interface, impedance mismatch
11.2.3 Exercising the internals
This section explains when and why acceptance tests can exercise business logic directly, and how they differ from unit tests and broader acceptance tests.
- Direct business logic testing: Describe how internals-oriented acceptance tests exercise specific business or application logic without going through a UI or external API, using validation rules as an example.
Why it matters: Direct tests let you examine a particular set of business rules without involving the user interface. - Precision and specification clarity: Explain why isolating business rules in acceptance tests can help the team and customer find specification problems.
Why it matters: Clear, focused examples make complex business rules easier to review and track. - Acceptance and unit test language: Distinguish internals-oriented acceptance tests from unit tests in terms of their audiences and vocabulary, even when they exercise similarly small pieces of code.
Why it matters: The language of a test should serve the people who need to understand its results. - Limits of selective testing: Explain why internals-oriented acceptance tests supplement rather than replace broader acceptance tests, and where they are especially useful.
Why it matters: Focused rule checks need broader acceptance tests alongside them to assess whether the system works as it should.
Key terms: exercising the internals, business logic, validation rules, business rules, high-precision tests, specification, acceptance tests, unit tests, business domain, language of the implementation
11.2.4 Stubbing out the irrelevant
This section explains how to replace less relevant components in acceptance tests while retaining focused tests of real integrations.
- Stubbing for testability: Explain how stubbing out the irrelevant replaces components in acceptance tests and why teams do it, using an example.
Why it matters: Replacements can make acceptance tests practical when real components are slow, inconvenient, or unavailable. - Supplementary integration testing: Explain why stubbed acceptance tests need to be supplemented with tests using real components, and describe the balance recommended here.
Why it matters: Fast tests with replacements cannot alone establish that the system works with its real dependencies. - Modularity and coverage confidence: Explain how modularizing replaceable components helps assess the coverage of tests and manage integration risk.
Why it matters: Clear component boundaries make it easier to reason about what the tests cover when the system is assembled.
Key terms: stubbing out the irrelevant, stubs, simulators, alternative implementations, supplementary strategy, integrations, stubbed system, modularize, risk-management, overall coverage
11.2.5 Testing backdoors
This section explains how testing backdoors give acceptance tests alternative access to a system and may eventually evolve into public functionality.
- Testing backdoor access: Explain what a testing backdoor is and how acceptance tests can use a database as one.
Why it matters: A backdoor can make an acceptance test feasible when the usual interface cannot provide the access it needs. - Backdoor evolution into functionality: Describe how a testing backdoor can evolve beyond its original testing purpose.
Why it matters: An interface created for testing can reveal a useful capability for the application itself.
Key terms: testing backdoor, database, system’s API, side effects, public API, monitoring, fault-management
11.3 Technology-specific considerations
This section introduces why technology matters when choosing how to connect acceptance tests to a system.
- Technology and connection strategy: Explain why the technology used by a system matters when choosing how to connect acceptance tests to it.
Why it matters: The system’s technology affects which testing connections are feasible and how much effort they require.
Key terms: acceptance tests, system under test, technology, strategy
11.3.1 Programming libraries
This section explains how to choose interfaces, test formats, and stubs for acceptance testing programming libraries.
- Library Acceptance Test Interface: Explain why testing a library through its public programming interface can be considered both going under the skin and testing end to end, and identify the remaining interface choice.
Why it matters: The library’s public API gives you a natural place to exercise its behavior from a user’s perspective. - Technical Test Expression Choices: Explain why library stories and acceptance tests can be technical, and describe the choice between Java-based tests and a more accessible format.
Why it matters: The audience and desired expressiveness help determine how technical your acceptance tests should be. - Stubbing Library Test Dependencies: Describe when stubbing can be useful in library acceptance tests, using the system clock or logging example.
Why it matters: Selective stubbing can make library acceptance tests easier to run.
Key terms: public programming interface, under the skin, end-to-end, public API, user stories, acceptance tests, JUnit, Fit, level of abstraction, stubbing
11.3.2 Faceless, distributed systems
This section weighs internal-API and end-to-end acceptance-testing strategies for faceless distributed systems.
- Hybrid distributed testing strategy: Explain why a distributed system might use mostly under-the-skin acceptance tests alongside a smaller set of end-to-end tests.
Why it matters: This strategy focuses most tests on functionality while still checking that the distributed parts work together. - Machine interface automation: Explain why end-to-end acceptance testing can be practical for faceless distributed systems.
Why it matters: The interface’s ease of automation can make testing through the network a reasonable choice. - Closed interface testing constraints: Describe how a proprietary closed interface can complicate acceptance testing and what alternatives the section identifies.
Why it matters: A closed interface can change whether testing directly through the network is practical.
Key terms: under the skin, internal APIs, end-to-end tests, machine-to-machine interfaces, end-to-end acceptance-testing strategy, network interface, proprietary, closed interface, vendor’s client-side API, client-side application’s UI, crawling under the skin
11.3.3 Console applications
This section compares ways to implement acceptance tests for console applications, favoring end-to-end tests through their typically simple command-line interfaces.
- CLI behavior and acceptance tests: Describe how command-line interactions map to acceptance tests, using the FTP upload and missing-file examples.
Why it matters: Mapping user-facing commands to behavior helps you identify concrete acceptance tests. - End-to-end CLI testing: Explain why testing a console application end-to-end through its UI is often practical and preferred.
Why it matters: A thin console interface can make realistic end-to-end acceptance tests relatively straightforward. - Alternative CLI testing strategies: Explain when testing below the console UI or using stubs may be useful, and what bypassing the UI leaves to test separately.
Why it matters: Alternatives to end-to-end testing can simplify particular checks, but they have coverage and maintenance tradeoffs.
Key terms: console applications, command-line interface, acceptance tests, put command, end-to-end strategy, console UI, semantic overhead, programming API, presentation, application logic
11.3.4 GUI applications
This section compares approaches to automating acceptance tests for desktop GUI applications and their practical tradeoffs.
- GUI testing complexity: Explain why desktop GUI acceptance tests are difficult to automate, including the challenges of timing and accessing components.
Why it matters: These difficulties help explain why a purely end-to-end GUI test may not be the best choice. - Coordinate based event testing: Describe how operating-system-level events can simulate GUI input and why coordinate-based testing is a last resort.
Why it matters: Simulating user input at the operating-system level can come at a high maintenance cost. - Native widget API tradeoffs: Explain how a native widget API supports black-box GUI testing and what limits that approach.
Why it matters: A native API can avoid blind coordinate targeting, but it does not eliminate the need for internal access. - Remote application testing hooks: Explain how exposed hooks give separate tests access beneath the UI, including their abstraction and cost tradeoffs.
Why it matters: Hooks can test functionality the existing UI cannot expose, but their design and maintenance require investment. - Same process test arrangements: Distinguish launching the application from tests from embedding tests in the application, and explain their practical tradeoffs.
Why it matters: The way tests and the application start affects automation, internal access, and stakeholder collaboration. - WebStart deployment testing example: Explain why the WebStart example embedded tests in the application and how those tests were invoked.
Why it matters: This example shows when the extra work of embedding tests can be worthwhile.
Key terms: semantic complexity, event-driven, multithreaded, UI components, operating system-level events, coordinate-based test automation, Swing components, native API, native widgets, black box
11.3.5 Web applications
This section compares ways to implement acceptance tests for web applications, including testing beneath the UI, simulating browsers, and controlling real browsers.
- Web UI testing tradeoffs: Explain the tradeoffs between testing a web application through its UI and testing beneath the web layer.
Why it matters: The test entry point affects both the web behavior exercised and the cost of running and maintaining tests. - Browser simulation capabilities: Describe how browser simulation libraries interact with a web application and why their convenience may not be sufficient for every application.
Why it matters: Simulation can make web tests convenient, but its support for the application's web technologies limits what those tests represent. - Real browser control tradeoffs: Explain what is needed to control a real browser in acceptance tests and the practical costs of doing so.
Why it matters: Real-browser testing can exercise actual browser behavior, but its automation and execution costs affect its suitability. - Team considerations in strategy: Explain why a team's skills, preferences, and collaboration matter when choosing a web acceptance-test strategy.
Why it matters: A technically viable test strategy also needs to work for the people who use and maintain it.
Key terms: end-to-end, actual HTTP interface, crawling under the application’s skin, service layer, simulation libraries, HTTP traffic, HTML, JavaScript, jWebUnit, real web browser
11.4 Tips for common problems
This section identifies acceptance test complexity and slow execution as problems that can make test suites difficult to maintain and delay feedback.
- Acceptance test design complexity: Explain why acceptance tests for complex systems need design and refactoring, like production code and unit tests.
Why it matters: Treating acceptance tests as code helps prevent the test suite itself from becoming an uncontrolled source of complexity. - Slow test feedback: Explain how long acceptance test execution delays feedback about changes that break other parts of a system.
Why it matters: Faster feedback helps a team find out sooner when a change has caused a problem elsewhere.
Key terms: acceptance tests, design, refactor, performance, test execution, end-to-end acceptance tests
11.4.1 Accelerating test execution
This section presents ways to accelerate acceptance test suites by reducing setup and persistence work, distributing execution, and weighing the trade-offs of shared state and faster resets.
- Test grouping for setup: Explain how grouping acceptance tests can reduce setup and restoration work.
Why it matters: Grouping tests by their actual needs can eliminate unnecessary setup without making them dependent on one another. - Reduced persistence disk work: Explain how changing persistence configuration can speed up acceptance tests.
Why it matters: Reducing disk activity can remove a major source of test-suite delay. - Distributed test execution patterns: Describe the three distributed execution patterns and explain why scaling target systems matters.
Why it matters: The right distribution pattern adds capacity where the work actually happens without introducing unnecessary complexity. - Dependent test setup trade-offs: Explain how shared setup can speed up tests and why dependencies make the suite harder to maintain and interpret.
Why it matters: Saving setup time can make changes and failure diagnosis much more expensive. - Transaction rollback for restoration: Explain how transaction rollback speeds database restoration and state its feasibility conditions.
Why it matters: Rollback can avoid repeated database rebuilds, but only when the system's transaction boundaries permit it. - Soft boot state reset: Describe a soft boot and explain its benefit and limitation.
Why it matters: Resetting only what is necessary can avoid a costly full restart while preserving a known starting state.
Key terms: setup, side effects, internal boundaries, cheating on persistence, disk I/O, in-memory mode, one-to-many, many-to-many, many-to-many-more, test client
11.4.2 Reducing complexity of test cases
This section explains how refactoring and reusable context objects can simplify acceptance tests, and how caching contexts can speed up repeated setup.
- Clear acceptance test structure: Explain how ordinary refactoring techniques reduce acceptance-test complexity and make tests communicate in the language of the user story.
Why it matters: Clear, cohesive tests make the behavior in a user story easier to see. - Reusable setup context objects: Describe how context objects simplify common acceptance-test setups and hide setup hierarchies from test cases.
Why it matters: A reusable context lets a test request its starting state without embedding or inheriting all the setup details. - Caching context state snapshots: Explain how a caching context object speeds up repeated setup and what support and limitation the section identifies.
Why it matters: Restoring a prepared session can avoid repeating slow setup requests.
Key terms: acceptance tests, user story, common fixtures, context objects, common setups, test cases, inheritance hierarchy, caching context object, session state, snapshot
11.4.3 Managing test data
This section presents guidelines for keeping acceptance-test data manageable through a shared data set, generation, and usable version control.
- Shared test data set: Explain why acceptance tests should share one data set and how that set can still be partitioned into manageable assets.
Why it matters: A shared but manageable data set avoids unnecessary duplication and synchronization work. - Generated test data: Explain why generating a data set can make it easier to manage, and describe how the target system's persistence implementation can help.
Why it matters: Generation can make a large data set easier to maintain and describe in familiar domain terms. - Diffable data set definitions: Explain why and how to version-control a test data set definition in a usable format.
Why it matters: A diffable history makes it easier to find out what test data changed between runs.
Key terms: single data set, partition, test data, generating test data, persistence implementation, domain classes, ORM, version-control, data-set definition, diff
11.5 Summary
Concepts explained in this chapter
- end-to-end testingEnd-to-end testing checks a system's behavior through its external interfaces across the interactions needed to deliver a feature.
- crawling under the skinCrawling under the skin is an acceptance-testing strategy that bypasses a system’s concrete user interface to test through an abstract interface or internal functionality.
- acceptance testingAcceptance testing is the process of checking whether a system satisfies conditions for acceptance by its stakeholders.
- dependent testA test whose execution or expected outcome depends on state or results produced by another test.
- parallel test executionThe concurrent execution of multiple software tests across available computing resources.
- soft bootA minimal system-reset operation that restores a system to a known state without performing a full restart.
- embedded testA test that is placed and executed within the application process rather than solely from an external test process.
- testing hookAn access point exposed by an application to let tests locate, operate on, or inspect its UI, logic, or state.
- browser simulation libraryA library that simulates browser interactions by exchanging HTTP traffic and interpreting returned web pages.
- abstract uiAn abstract UI is a test-facing interface that represents user interactions without depending on concrete, technology-specific widgets.
- datasetA dataset is a collection of related data values treated together for comparison or analysis.
- evaluation suiteA test suite is a collection of tests that can be run together to check software behavior.
- machine-to-machine interfaceA machine-to-machine interface is an interface through which software systems communicate without a person directly interacting with it.
- user storyA user story is a concise description of a software requirement from the perspective of a user or customer.
- white-box testingWhite-box testing is a testing approach that uses knowledge of a system's internal structure or implementation.
- black-box testingBlack-box testing checks a system through its externally visible behavior without relying on knowledge of its internal implementation.
- component hierarchyA component tree is the hierarchical arrangement of nested user-interface components that make up a page or interface.
- domain-specific languageA domain-specific language is a language designed to express concepts and operations within a particular problem domain.
- stubA stub is a test double that supplies predetermined responses to calls made by the system under test.
- testabilityThe degree to which software can be readily instantiated, exercised, and verified through tests.
- software testingSoftware testing is the evaluation of software to identify defects or assess whether it behaves as expected.
- automated testingAn automated test is a software test executed by a computer according to predefined instructions rather than performed manually.
- regression testA software test that checks whether previously working functionality remains intact after changes.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
