Flaky test
A flaky test is an automated test whose outcome varies across executions under equivalent conditions without a relevant change to the software or test intent.
The Art of Unit Testing 3e
Test flakiness is the tendency of tests to produce inconsistent pass-or-fail results without a relevant code change.
- How it is used
- Builds trust in test results by distinguishing unpredictable failures from real production defects.
- How deep it goes
- Defines inconsistent outcomes, traces dependency and state causes, and gives a fix-convert-kill response process.
- Examples
- Uses a Jest verifier with beforeEach setup, dynamic values, databases, networks, and a stubbed network endpoint.
- What you can do after
- Diagnose/debug with it by identifying inconsistent tests, isolating them, controlling dependencies, converting levels, or removing them.
100 Java Mistakes and How to Avoid Them
A flaky test is an automated test whose result varies between runs despite unchanged code and inputs.
- How it is used
- Uses flaky tests to explain how unstable external state undermines continuous integration.
- How deep it goes
- Explains timing and filesystem race conditions with focused Java snippets, then stops at practical avoidance guidance.
- Examples
- Uses a backup-folder timestamp assertion and filesystem file-size checks to demonstrate changing state between operations.
- What you can do after
- Diagnose timing and state-change failures, then stabilize tests by reusing observations and reducing separate filesystem queries.
Read how 100 Java Mistakes and How to Avoid Them explains flaky test →
Related concepts
- bugkind of · A software bug is a defect in software or a related system component that causes behavior to differ from what is intended or expected.
- trustcontrasts with · Trust is confidence that a person, organization, system, or source is reliable and will act or behave appropriately in a given context.
- provider stateaddressed by · A provider state is a named precondition describing the provider data or runtime situation required before an interaction is verified.
- dependencyuses · A software dependency is a component, resource, or prerequisite on which another software component or operation relies to function or proceed.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.