Chapter 12: Integration Testing
The introduction explains why integration tests matter for software that interacts with the world and why they are difficult to make reliable.
After this chapter you will be able to
- Reach of integration tests. Explain why software that interacts with the world needs integration tests in addition to unit tests.
- Sources of integration test difficulty. Describe why integration testing is hard when tests interact with systems outside our direct control and with real-world failures.
- Integration Tests and Real Dependencies. Explain how the section defines an integration test and how it differs from a unit test that uses mocks.
- Dependency Behavior and Visibility. Explain what integration tests verify about external dependencies and why that makes them hard to maintain.
- Visible input and outcome links. Explain why an integration test should show how its inputs relate to its outputs or side effects, without duplicating low-level business-logic tests.
- Generated file test inputs. Describe why the file-upload test should generate its file from data in the test rather than rely on a hard-coded file.
- Dependency-enforced state invariants. Explain why task-state transition rules must be enforced by the database and tested against the real dependency rather than trusted to application code or mocks.
- Ordering of task transitions. Describe how the section expresses valid task-state transitions as an ordering: empty precedes any state, a state cannot transition to itself, and terminal states have no successors.
- Algebraic model as test oracle. Explain how to use the transition ordering to inspect every possible case and check whether the real database upholds the intended invariants.
- Sequential Tests Versus Real Load. Explain why a sequential arrange-act-assert integration test can pass while the deployed system still fails under concurrent use, as illustrated by gaps in inventory short codes.
- Concurrent Tests and Invariant Checks. Describe the concurrent testing approach in the section: let multiple threads exercise the code with minimal prescribed sequencing, then check the invariant after they finish. Explain what this approach can and cannot establish.
- Pruning redundant integration tests. Explain why useful early integration tests may later be deleted when broader tests cover the same behavior, and why keeping redundant tests is costly.
- Isolation as the reliability requirement. Explain why integration tests need isolation even when failures and incomplete cleanup are unavoidable.
- On-demand real infrastructure. Describe how provisioning real dependencies for a test avoids shared-infrastructure interference, and why mocks alone do not establish that an integration works.
- Randomized resource namespaces. Explain why resource names are randomized even when tests create and attempt to remove their own infrastructure.
- Tiered names for cleanup. Describe why randomized resources also need a tiered naming scheme that supports cleanup at different levels.
- Integration Testing Tradeoff. Explain why a reliable integration test suite is difficult to build and what makes the effort worthwhile.
What it builds on
- Use data to show what’s under test (chapter 11) · Using data to show what a test is about
- Data makes tests flexible (chapter 11) · Using data to make tests flexible
- Ordering as a property (chapter 7) · The idea of ordering as a property beyond sorting.
- Using randomized data in tests (chapter 11) · The comparison to randomized tests, which also explore cases not specified one by one.
Section by section
Introduction
The introduction explains why integration tests matter for software that interacts with the world and why they are difficult to make reliable.
- Reach of integration tests: Explain why software that interacts with the world needs integration tests in addition to unit tests.
Why it matters: This explains what integration tests add when software interacts with the world. - Sources of integration test difficulty: Describe why integration testing is hard when tests interact with systems outside our direct control and with real-world failures.
Why it matters: These difficulties explain why reliability is a particular concern for integration test suites.
Key terms: integration tests, unit test, edges of our system, integration testing, unreliable networks, service outages, race conditions, data corruption
12.1 Writing Integration Tests
Defines integration tests by their use of I/O, shows that they can take different shapes, and explains why testing real dependencies is valuable but hard to maintain.
- Integration Tests and Real Dependencies: Explain how the section defines an integration test and how it differs from a unit test that uses mocks.
Why it matters: The use of real dependencies changes both what the test can verify and how it must be approached. - Flexible Integration Test Shapes: Describe the two test structures the section presents and why neither is required for every integration test.
Why it matters: A reader need not force every integration test into one fixed sequence. - Dependency Behavior and Visibility: Explain what integration tests verify about external dependencies and why that makes them hard to maintain.
Why it matters: Real-system checks can reveal mistaken assumptions, but the behavior they examine is less visible in the test itself.
Key terms: integration test, I/O, real dependencies, mocks, arrange, act, assert pattern, assert as you go, dependencies, distance, data-oriented approach
12.2 Drive your integration tests with data
The section shows how generating integration-test inputs from data makes tests easier to understand and the suite easier to adapt.
- Visible input and outcome links: Explain why an integration test should show how its inputs relate to its outputs or side effects, without duplicating low-level business-logic tests.
Why it matters: A visible cause-and-effect story makes an integration test easier to maintain. - Generated file test inputs: Describe why the file-upload test should generate its file from data in the test rather than rely on a hard-coded file.
Why it matters: Generated inputs make it clearer what a failing file-upload test was actually exercising. - Reusable test data tooling: Justify the extra work of building data-generation support for an integration-test suite, including how it helps vary test inputs.
Why it matters: The upfront support code pays off when tests need new inputs or system states.
Key terms: integration tests, inputs, outputs, side-effects, hard coded file, provenance, generate, assertions, test data, support code
12.3 Impose algebraic properties on your dependencies
The section uses an algebraic model of task-state transitions to define and test the behavior a database must enforce.
- Dependency-enforced state invariants: Explain why task-state transition rules must be enforced by the database and tested against the real dependency rather than trusted to application code or mocks.
Why it matters: A passing unit test does not establish that the database enforces the rule your service relies on. - Ordering of task transitions: Describe how the section expresses valid task-state transitions as an ordering: empty precedes any state, a state cannot transition to itself, and terminal states have no successors.
Why it matters: The ordering makes the transition rules precise enough to check against the database. - Algebraic model as test oracle: Explain how to use the transition ordering to inspect every possible case and check whether the real database upholds the intended invariants.
Why it matters: A clear model helps reveal mistakes when business rules are translated into a database’s expression language.
Key terms: invariants, integration testing, dependencies, race conditions, ordering, irreflexive binary relation, terminal states, state machine, algebraic model
12.4 Embrace concurrency, beat up your software, and find bugs
This section explains why sequential integration tests can miss bugs that appear under load and how concurrent tests can expose them.
- Sequential Tests Versus Real Load: Explain why a sequential arrange-act-assert integration test can pass while the deployed system still fails under concurrent use, as illustrated by gaps in inventory short codes.
Why it matters: A passing sequential test may not cover the conditions in which a production bug occurs. - Concurrent Tests and Invariant Checks: Describe the concurrent testing approach in the section: let multiple threads exercise the code with minimal prescribed sequencing, then check the invariant after they finish. Explain what this approach can and cannot establish.
Why it matters: Running competing actions can expose failures that carefully scripted examples never trigger.
Key terms: example-based testing, arrange, act, assert, short codes, concurrently, concurrent tests, invariant, threads, determinism
12.5 Regularly prune redundant tests!
Explains why an integration test suite should shed tests that become redundant as the project matures.
- Pruning redundant integration tests: Explain why useful early integration tests may later be deleted when broader tests cover the same behavior, and why keeping redundant tests is costly.
Why it matters: A test suite stays useful when each costly integration test has a distinct reason to remain.
Key terms: integration tests, anti-regression layer, invariants, higher-level API, redundant tests
12.6 Building a reliable integration test suite
The section shows how to make integration tests reliable by isolating their infrastructure, including when tests fail and leave resources behind.
- Isolation as the reliability requirement: Explain why integration tests need isolation even when failures and incomplete cleanup are unavoidable.
Why it matters: Isolation keeps a flaky outside world from making the test suite’s findings untrustworthy. - On-demand real infrastructure: Describe how provisioning real dependencies for a test avoids shared-infrastructure interference, and why mocks alone do not establish that an integration works.
Why it matters: Testing against isolated real services can reveal integration problems without tests fighting over shared resources. - Randomized resource namespaces: Explain why resource names are randomized even when tests create and attempt to remove their own infrastructure.
Why it matters: A fresh namespace prevents a failed cleanup from contaminating the next run of a test. - Tiered names for cleanup: Describe why randomized resources also need a tiered naming scheme that supports cleanup at different levels.
Why it matters: Recoverable naming keeps abandoned test infrastructure identifiable and prevents it from accumulating costs.
Key terms: integration test, isolation, clean slate, on demand, isolated, ephemeral, actual services, randomization, namespace, stale resources
12.7 Wrapping up
This section closes the chapter by explaining why reliable integration tests are worth the difficulty and encouraging data-focused experimentation.
- Integration Testing Tradeoff: Explain why a reliable integration test suite is difficult to build and what makes the effort worthwhile.
Why it matters: The difficulty is justified when tests find bugs that might otherwise reach production. - Data Focused Test Experimentation: Describe the section’s recommended approach to improving integration tests rather than relying only on a fixed Arrange, Act, Assert pattern.
Why it matters: A data-focused approach leaves room to try test patterns suited to the behavior being checked.
Key terms: integration tests, unit tests, correctness, data, Arrange, Act, Assert, randomization, concurrently
12.8 Summary
Concepts explained in this chapter
- integration testIntegration testing evaluates interactions between multiple software components or external dependencies.
- test suiteA test suite is a collection of automated or executable tests that assess software against specified expectations.
- randomizationRandomization is the generation or selection of varying test inputs to exercise a broader range of program states than fixed examples.
- reliabilitySoftware reliability is the likelihood that software continues to perform required functions correctly under stated conditions.
- resource namingResource naming is the practice of assigning structured identifiers to external resources so they can be grouped, located, and removed systematically.
- test isolationIsolation is execution of a test without interaction with its external or substituted dependencies.
- algebraic lawAn algebraic property is a characteristic describing how an operation behaves under operand order, grouping, and relationships between inputs and outputs.
- Order propertyAn ordering is a binary relation that arranges or compares elements according to specified properties such as reflexivity, transitivity, and antisymmetry.
- accuracyCorrectness is the property that an implementation produces the behavior and results specified by its intended semantics and requirements.
- business ruleA business rule is a domain constraint or policy that determines valid actions, states, or outcomes.
- constraintA constraint is a condition that restricts which data representations or program states are valid.
- invariantAn invariant is a condition that must remain true across the valid states of a data structure or computation.
- provenanceProvenance is information describing the origin and sequence of steps that produced a result or movement of data.
- data-oriented programmingData-oriented programming is a programming approach that organizes software around explicit data representations and operations on those representations.
- dependencyA dependency is a relationship in which one software element relies on another element's data, behavior, or availability.
- mockingA mock is a test-controlled replacement or implementation that stands in for an object or function and allows its behavior or interactions to be controlled.
- hardcodingHard coding is embedding environment-specific values or infrastructure assumptions directly in application code.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
