A flaky test produces different results across identical executions when the code and environment have not changed. It weakens build feedback: developers may rerun builds, batch changes, or learn to ignore failures. A red build then stops reliably meaning “stop and investigate.” section 5.4 section 5.4.1
Look for both the source of variability and the costs it spreads through the workflow:
- Timing assumptions, such as system time or background work completing later than expected, can make results intermittent. section 5.4.1
- Network latency or shared test data can also make outcomes vary without a code change. section 5.4
- Reruns consume CI time; retries, sleeps, and ignored failures can mask defects and make unreliable signals part of normal workflow. section 5.4.1
A time-coupled expiration test
A test creates an order, advances time by 30 minutes, processes expirations, and checks that the order expired. If a background worker runs late under load, the assertion may fail in CI but pass on a rerun. The changing timing—not a changed assertion—makes the result unreliable. section 5.4.1
Listing 5.3 A time-coupled expiration test
def test_order_expires_after_30_minutes():
order = create_order()
advance_time(minutes=30)
process_expirations()
assert order.status == "expired"The listing shows how the test couples its assertion to a timed expiration workflow. section 5.4.1
Retrying can hide the cause
An intermittent user lookup may tempt a developer to add retries. If the test helper starts an asynchronous database insert without awaiting it, the lookup races the write; awaiting the insert addresses the identified cause, while retries can conceal it. section 8.4



