An optimization can look simpler and faster while adding overhead that erases its gains. JMH (Java Microbenchmark Harness) measures isolated Java code, down to individual methods, so competing implementations can be tested against explicit workload assumptions. section 7.1.1
Choose inputs that reflect expected use. A benchmark with an assumed 100,000 accounts answers a different question from one with 10,000. Handwritten timing code also risks misleading results through inadequate warmup, JIT optimization, or loop unrolling; using JMH is a more disciplined starting point. section 7.1.2
Sequential versus parallel lookup
For 10,000 accounts, JMH compares a sequential stream with a parallel stream. The benchmark consumes each result with a Blackhole. The reported average times are 0.027 ± 0.002 ms/op and 0.030 ± 0.002 ms/op, respectively: nearly identical, without a demonstrated benefit that justifies the parallel version’s coordination, monitoring, and debugging costs. section 7.1.1
Listing 7.4 Implementing the account’s benchmark logic
@Benchmark
public void baseline(Blackhole blackhole) {
Optional<Account> account =
new AccountFinder(ACCOUNTS).account(random.nextInt(10_000)); #A
blackhole.consume(account); #B
}
@Benchmark
public void parallel(Blackhole blackhole) {
Optional<Account> account =
new AccountFinder(ACCOUNTS).accountOptimized(random.nextInt(10_000)); #C
blackhole.consume(account);
}
Inspect the two benchmark methods and how they pass lookup results to a Blackhole. section 7.1.1
A more elaborate comparison can parameterize account count and specify warmup, measurement, forks, and threads. Inspect these settings and the workload rather than trusting generated benchmark code or its assumptions automatically. section 7.1.2
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Benchmark)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@Threads(1)
public class AccountFinderJMHBenchmark {
@Param({"1000", "10000", "100000"})
private int accountCount;
private AccountFinder accountFinder;
private List<Account> accounts;
private List<Integer> searchIds;
private Random random;
@Setup(Level.Trial)
public void setupTrial() {
random = new Random(42); #A
accounts = generateAccounts(accountCount);
accountFinder = new AccountFinder(accounts);
searchIds = generateRandomIds(random, 100);
}See a generated benchmark configuration with three account counts and explicit warmup and measurement settings. section 7.1.2
File lookup versus caching
A word-existence benchmark compares the original file-based implementation with a cached implementation. The comparison helps evaluate a proposed optimization, but eager caching also introduces memory costs that lookup timings alone do not settle. section 7.4.1
Use JMH to compare isolated code paths, not as a substitute for end-to-end testing. Gatling can measure the whole system under request traffic, helping assess whether an isolated improvement matters at the endpoint level. section 7.2.3
