Chapter 7: Chapter 7. Test-driving the unpredictable
This introduction previews the challenges of test-driving time-dependent and multithreaded Java code.
After this chapter you will be able to
- Current time testing challenge. Explain why code whose output or behavior depends on the current time poses a testing challenge.
- Timestamp Dependency in Tests. Explain why a hard-coded timestamp makes the naïve logging test unlikely to pass.
- Time Dependency Workarounds. Compare passing time as a parameter with overriding a time-dependent method, and explain why the section prefers a more generic solution.
- System time abstraction. Explain how SystemTime replaces direct calls to Java’s time APIs while preserving real-time behavior by default.
- Configurable time source strategy. Describe how SystemTime selects a TimeSource and why the interface permits more than a fixed test clock.
- Controlled time in tests. Explain how the log-formatter test uses a fixed time and how production code accesses it. Describe why the fake time source must be removed after the test.
- Timestamp value and format. Explain why the fixed-time test alone may not verify the required timestamp format, and how a separate format test complements it. Distinguish checking the format from checking the actual time used.
- Sequential first testing strategy. Describe the section’s recommended order for test-driving functionality and concurrent behavior, including the benefit and caveat of isolating sequential logic.
- Concurrent testing strategy. Explain why concurrent code needs tests beyond sequential behavior and how to approach its wider error-situation space.
- Desired and undesired behavior. Distinguish desired concurrent functionality from unwanted concurrent situations, giving an example of each.
- Deadlock starvation and liveness. Distinguish deadlock from starvation and explain how both affect liveness.
- Thread safety and interleaving. Explain what thread-safety means and why concurrent execution can threaten it.
- Limits of concurrency testing. Explain why thread-safety tests provide confidence rather than proof and how intelligent brute force can improve that confidence without controlling execution inside the code under test.
- Shared counter concurrency test. Describe the shared counter test, the defect it exposes, and the implementation change prompted by the improved test.
- Barrier based test coordination. Describe how the improved counter test increases concurrency and uses entry and exit barriers to coordinate the worker threads and test method.
- Testing blocking and interruption. Describe how the ticket example tests that a method blocks and then responds to interruption.
- Unreliable thread state observations. Explain why Thread#getState is not a reliable way to test whether a method blocks.
- Thread factory injection. Explain how a ThreadFactory and dependency injection make thread creation observable in a test.
- Thread lifecycle latches. Describe how the custom thread lets other threads wait for its start and completion.
- Server lifecycle test. Trace how the example test verifies the server's background thread starts and stops.
- Fixed waiting tradeoff. Explain why sleeping before checking an asynchronous result can cause unreliable or unnecessarily slow tests.
- Retried assertion polling. Describe how RetriedAssert checks asynchronous results and explain its timeout tradeoff.
- Synchronization testing approach. Describe how to test multithreaded collaboration using invariants and divide-and-conquer.
- Standard synchronization abstraction rationale. Explain why the section recommends Java's standard interthread communication abstractions even though shared-nothing is the ideal.
- Semaphore permit lifecycle. Explain how threads acquire and release semaphore permits and why permits become available to other threads.
- Counting and single-permit semaphores. Distinguish a counting semaphore from a single-permit semaphore, using the download example to explain the concurrency limit.
- Latch synchronization and countdown. Explain how a latch blocks threads and how CountDownLatch releases them after a predetermined number of countdown calls. Describe its use in waiting for parallel threads to reach a point in their execution.
- Reusable barrier coordination. Explain how a CyclicBarrier coordinates a group of threads across repeated stages, and distinguish it from a latch.
- Two-thread data exchange. Describe how an Exchanger coordinates two threads exchanging data, using the reader and writer buffer example.
- Future result synchronization. Explain how a Future is shared between a calling thread and a background computation, and how the caller obtains the result.
What it builds on
- Test doubles stand in for dependencies (chapter 4) · Test doubles can stand in for dependencies.
- Inject dependencies (chapter 4) · Dependencies can be supplied to the code that uses them.
- Test-code-refactor: the heartbeat (chapter 1) · The test-code-refactor approach to developing functionality.
- Isolate dependencies (chapter 4) · Isolating dependencies to support testable designs.
Section by section
Introduction
This introduction previews the challenges of test-driving time-dependent and multithreaded Java code.
7.1 Test-driving time-based functionality
This section introduces the testing challenge posed by code whose behavior depends on the current time.
- Current time testing challenge: Explain why code whose output or behavior depends on the current time poses a testing challenge.
Why it matters: Recognizing this dependency helps you see why time-based behavior needs special attention in tests.
Key terms: current time, time-based functionality
7.1.1 Example: logs and timestamps
The section uses a logging example to show why system time makes a naïve test unreliable and why a reusable way to fake time is preferable to local workarounds.
- Timestamp Dependency in Tests: Explain why a hard-coded timestamp makes the naïve logging test unlikely to pass.
Why it matters: Recognizing the clock dependency helps you see why an otherwise straightforward output assertion is unreliable. - Time Dependency Workarounds: Compare passing time as a parameter with overriding a time-dependent method, and explain why the section prefers a more generic solution.
Why it matters: The trade-off explains why a fix for one test may not be the best approach across a codebase.
Key terms: timestamp, system clock, current system time, method parameter, overridden method, generic solution, system-time access
7.1.2 Abstracting system time
This section develops a configurable system-time abstraction that lets tests substitute controlled time sources for the real clock.
- System time abstraction: Explain how SystemTime replaces direct calls to Java’s time APIs while preserving real-time behavior by default.
Why it matters: A common time abstraction gives production code a single place to obtain time without changing its default behavior. - Configurable time source strategy: Describe how SystemTime selects a TimeSource and why the interface permits more than a fixed test clock.
Why it matters: Swappable time sources let tests control time without permanently hard-coding it into production behavior. - Time source test isolation: Describe the tests for default and configured time and explain why teardown resets the source.
Why it matters: Restoring the clock after a test prevents a fake time source from contaminating the rest of the suite.
Key terms: SystemTime, System#currentTimeMillis, Calendar#getInstance, TimeSource, Strategy design pattern, teardown, default TimeSource, fixed time
7.1.3 Testing log output with faked system time
The section makes a log-formatter test time-independent using a fake time source and examines how to verify both the timestamp’s value and its format.
- Controlled time in tests: Explain how the log-formatter test uses a fixed time and how production code accesses it. Describe why the fake time source must be removed after the test.
Why it matters: Controlling time makes the test repeatable without contaminating other tests. - Timestamp value and format: Explain why the fixed-time test alone may not verify the required timestamp format, and how a separate format test complements it. Distinguish checking the format from checking the actual time used.
Why it matters: A passing timestamp test should not conceal either a wrong format or a wrong time.
Key terms: SystemTime, TimeSource, SystemTime#asDate, fake system time, DateFormat, common log format, regular expression, timestamp
7.2 Test-driving multithreaded code
This section explains how concurrency relates to sequential execution and recommends test-driving sequential logic before verifying concurrent behavior.
- Concurrent and sequential execution: Explain how the section describes concurrent threads as appearing to run simultaneously while each individual thread executes sequentially, and relate this to testing.
Why it matters: This helps you see why familiar testing techniques still apply when a program uses multiple threads. - Sequential first testing strategy: Describe the section’s recommended order for test-driving functionality and concurrent behavior, including the benefit and caveat of isolating sequential logic.
Why it matters: Separating the logic lets you test much of the functionality simply without overlooking its effects on concurrency.
Key terms: concurrent programming, thread scheduler, sequential programs, sequential logic, concurrent behavior, synchronization
7.2.1 What are we testing for?
This section explains what concurrency tests should target and why nondeterministic execution makes exhaustive testing impractical.
- Concurrent testing strategy: Explain why concurrent code needs tests beyond sequential behavior and how to approach its wider error-situation space.
Why it matters: This helps you choose useful concurrency tests without assuming that every possible error can be covered. - Test induced Heisenbugs: Describe how testing or debugging can hide a concurrency bug.
Why it matters: A disappearing bug does not necessarily mean the underlying concurrency problem is fixed. - Desired and undesired behavior: Distinguish desired concurrent functionality from unwanted concurrent situations, giving an example of each.
Why it matters: Both producing the right behavior and avoiding harmful states matter when testing concurrent code. - Deadlock starvation and liveness: Distinguish deadlock from starvation and explain how both affect liveness.
Why it matters: Recognizing these distinct failures helps you identify ways concurrent programs can stop making progress.
Key terms: concurrent behavior, error-situation space, repeated, long test runs, Heisenbugs, asynchronous execution, deadlock, starvation, liveness, thread scheduler
7.2.2 Thread-safety
This section explains thread-safety testing limitations and uses a shared counter to demonstrate how synchronized test starts and finishes can make concurrency defects easier to expose.
- Thread safety and interleaving: Explain what thread-safety means and why concurrent execution can threaten it.
Why it matters: Understanding interleaving helps explain why code that works in one thread may fail with several. - Limits of concurrency testing: Explain why thread-safety tests provide confidence rather than proof and how intelligent brute force can improve that confidence without controlling execution inside the code under test.
Why it matters: A passing concurrency test can be misleading, especially if the test changes the behavior it tries to observe. - Shared counter concurrency test: Describe the shared counter test, the defect it exposes, and the implementation change prompted by the improved test.
Why it matters: The counter makes an otherwise elusive concurrency failure visible as a mismatch in the final value. - Barrier based test coordination: Describe how the improved counter test increases concurrency and uses entry and exit barriers to coordinate the worker threads and test method.
Why it matters: Coordinating the test around the work makes critical interleavings more likely and ensures the result is checked after workers finish.
Key terms: thread-safe, concurrently, byte-code instructions, properties, invariants, intelligent brute-force test, false negatives, thread scheduling, Counter, increment
7.2.3 Blocking operations
This section explains how to test a blocking method and why thread state is unreliable evidence of blocking.
- Purpose of blocking operations: Explain what it means for a method to block until an event has occurred, using an example from the section.
Why it matters: Blocking is useful when a caller must not proceed before a required event or operation completes. - Testing blocking and interruption: Describe how the ticket example tests that a method blocks and then responds to interruption.
Why it matters: A blocking test needs to check the wait without leaving its worker thread stuck. - Unreliable thread state observations: Explain why Thread#getState is not a reliable way to test whether a method blocks.
Why it matters: A thread-state check can mislead a test about whether blocking behavior is correct.
Key terms: blocking method, commit, BlackMarket#buyTicket, sellTicket, InterruptedException, Thread#sleep, Thread#getState, waiting state, spurious wakeups, Object#wait
7.2.4 Starting and stopping threads
This section shows how to test that a server starts and stops its background thread using an injected ThreadFactory and observable thread lifecycle signals.
- Thread factory injection: Explain how a ThreadFactory and dependency injection make thread creation observable in a test.
Why it matters: Injecting the factory lets the test observe the thread the server actually uses. - Thread lifecycle latches: Describe how the custom thread lets other threads wait for its start and completion.
Why it matters: Lifecycle signals let the test wait for observable thread events rather than merely assume they happened. - Server lifecycle test: Trace how the example test verifies the server's background thread starts and stops.
Why it matters: The sequence checks both that the background thread starts and that stopping the server causes it to finish.
Key terms: ThreadFactory, dependency injection, Thread, latches, run, waitForStarted, waitForStopped, Server, start latch, stop latch
7.2.5 Asynchronous execution
This section explains ways to test work performed asynchronously, including isolating typed Runnables and waiting or polling for results.
- Typed Runnable isolation: Explain how typed Runnables can split testing asynchronous work into two separate problems.
Why it matters: This gives you a way to check what background work does without testing everything together. - Fixed waiting tradeoff: Explain why sleeping before checking an asynchronous result can cause unreliable or unnecessarily slow tests.
Why it matters: Recognizing the tradeoff helps you avoid both flaky tests and unnecessary delays. - Retried assertion polling: Describe how RetriedAssert checks asynchronous results and explain its timeout tradeoff.
Why it matters: Polling can finish a successful test sooner than a generous fixed sleep while still bounding the wait.
Key terms: typed Runnable, ThreadFactory, asynchronous service, wait time, RetriedAssert, polling, assertion, timeout
7.2.6 Synchronization between threads
This section explains thread synchronization and a divide-and-conquer approach to testing collaboration between threads.
- Thread collaboration and synchronization: Explain why independently executing worker threads sometimes need synchronization.
Why it matters: Recognizing where independent work depends on another thread helps identify what needs testing. - Synchronization testing approach: Describe how to test multithreaded collaboration using invariants and divide-and-conquer.
Why it matters: This approach makes a complex collaboration more manageable to test.
Key terms: worker threads, synchronization point, invariants, divide-and-conquer, test doubles, synchronization problems
7.3 Standard synchronization objects
This section recommends standard Java concurrency abstractions for necessary interthread synchronization instead of constructing solutions from low-level primitives.
- Standard synchronization abstraction rationale: Explain why the section recommends Java's standard interthread communication abstractions even though shared-nothing is the ideal.
Why it matters: Standard abstractions help address unavoidable synchronization needs without reinventing low-level solutions.
Key terms: shared-nothing, interthread communication, java.util.concurrent, low-level locks, wait/notify
7.3.1 Semaphores
This section explains how semaphore permits limit concurrent access to work or shared resources.
- Semaphore permit lifecycle: Explain how threads acquire and release semaphore permits and why permits become available to other threads.
Why it matters: The permit lifecycle explains how a semaphore controls access among threads. - Counting and single-permit semaphores: Distinguish a counting semaphore from a single-permit semaphore, using the download example to explain the concurrency limit.
Why it matters: The number of permits determines how many threads can proceed with protected work at once.
Key terms: semaphore, permit, acquire, release, counting semaphore, single permit, synchronization lock, shared resource
7.3.2 Latches
This section explains latches as one-use synchronization points and describes how CountDownLatch releases waiting threads.
- Latch synchronization and countdown: Explain how a latch blocks threads and how CountDownLatch releases them after a predetermined number of countdown calls. Describe its use in waiting for parallel threads to reach a point in their execution.
Why it matters: A latch provides a way to hold threads at a synchronization point until the required work has reached that point.
Key terms: latch, CountDownLatch, countdown calls, synchronization points
7.3.3 Barriers
This section explains reusable barriers for coordinating threads and the Exchanger for swapping data between two threads.
- Reusable barrier coordination: Explain how a CyclicBarrier coordinates a group of threads across repeated stages, and distinguish it from a latch.
Why it matters: A reusable barrier lets a group of threads advance through stages together. - Two-thread data exchange: Describe how an Exchanger coordinates two threads exchanging data, using the reader and writer buffer example.
Why it matters: An Exchanger provides a meeting point where two threads can hand data to each other.
Key terms: barrier, CyclicBarrier, latch, await, Exchanger, buffers
7.3.4 Futures
This section explains how a Future lets a calling thread obtain the result of a cancellable asynchronous task.
- Future result synchronization: Explain how a Future is shared between a calling thread and a background computation, and how the caller obtains the result.
Why it matters: A Future provides a way to coordinate a background result without building your own synchronized callback.
Key terms: Future, FutureTask, get, asynchronous tasks, timeout
7.4 Summary
Concepts explained in this chapter
- thread safetyThread safety is the property of software that behaves correctly when accessed by multiple threads concurrently.
- concurrencyConcurrency is the overlapping progress of multiple computations.
- thread synchronizationThread synchronization is the coordination of concurrent threads to control access to shared state or the ordering of their operations.
- intelligent brute-force testAn intelligent brute-force test deliberately maximizes the likelihood of exercising error-producing execution conditions.
- blockingBlocking behavior occurs when an operation waits for a condition or event before proceeding.
- test-driven developmentTest-driven development is a software development technique in which tests are written before the code they guide and verify.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
