liveBook · Manning

Flaky test

As explained in 100 Java Mistakes and How to Avoid Them

A flaky test is an automated test whose result varies between runs despite unchanged code and inputs.

A flaky test produces different results across runs even when code and inputs are unchanged. Timing-sensitive checks can pass for months, then fail when a boundary changes, disrupting continuous integration and wasting developers’ time. section 9.11

Comparing against a second clock reading

A backup-folder test calls createBackupFolder(), then calls LocalDateTime.now() again to construct the expected name. If the minute changes between calls, the assertion fails. Capture the relevant time once and use that value for both production behavior and the assertion. section 9.11

Reduce instability by avoiding assumptions that the world remains unchanged between related operations:

  • Minimize separate filesystem queries when several attributes are needed; Files.readAttributes() obtains them together. section 9.11
  • Prefer one operation that checks and uses filesystem state together when intervening changes could invalidate the result. section 9.11
@Test
public void testBackupFolder() throws IOException {
  Path tempDir = Files.createTempDirectory("test");
  Path backupFolder = createBackupFolder(tempDir);     #1
  var nameFormat = DateTimeFormatter.ofPattern(
    "'Backup'-yyyy-MM-dd-HH-mm");
  String expectedName = nameFormat.format(LocalDateTime.now());
  assertEquals(backupFolder.getFileName().toString(),
    expectedName);
}

Inspect the time-dependent assertion that makes the backup-folder test flaky. section 9.11

How 100 Java Mistakes and How to Avoid Them covers it

How deep it goes
Explains timing and filesystem race conditions with focused Java snippets, then stops at practical avoidance guidance.
Examples
Uses a backup-folder timestamp assertion and filesystem file-size checks to demonstrate changing state between operations.
What it assumes
Assumes familiarity with Java tests, date-time APIs, filesystem operations, and assertions.
What you can do after
Diagnose timing and state-change failures, then stabilize tests by reusing observations and reducing separate filesystem queries.
How it connects
Connects flaky tests with continuous integration, filesystem queries, atomic operations, and time-dependent behavior.

Also explained in

Compare how 2 books explain flaky test →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of 100 Java Mistakes and How to Avoid Them. The section links point to the parts of the book it draws on.