liveBook · Manning

JMH

As explained in Vibe Engineering

JMH is a Java harness for constructing and running reliable microbenchmarks of Java code.

An optimization can look simpler and faster while adding overhead that erases its gains. JMH (Java Microbenchmark Harness) measures isolated Java code, down to individual methods, so competing implementations can be tested against explicit workload assumptions. section 7.1.1

Choose inputs that reflect expected use. A benchmark with an assumed 100,000 accounts answers a different question from one with 10,000. Handwritten timing code also risks misleading results through inadequate warmup, JIT optimization, or loop unrolling; using JMH is a more disciplined starting point. section 7.1.2

Sequential versus parallel lookup

For 10,000 accounts, JMH compares a sequential stream with a parallel stream. The benchmark consumes each result with a Blackhole. The reported average times are 0.027 ± 0.002 ms/op and 0.030 ± 0.002 ms/op, respectively: nearly identical, without a demonstrated benefit that justifies the parallel version’s coordination, monitoring, and debugging costs. section 7.1.1

Listing 7.4 Implementing the account’s benchmark logic
@Benchmark
public void baseline(Blackhole blackhole) {
Optional<Account> account =
  new AccountFinder(ACCOUNTS).account(random.nextInt(10_000)); #A
blackhole.consume(account); #B
}
 
@Benchmark
public void parallel(Blackhole blackhole) {
Optional<Account> account =
  new AccountFinder(ACCOUNTS).accountOptimized(random.nextInt(10_000)); #C
blackhole.consume(account);
}
 

Inspect the two benchmark methods and how they pass lookup results to a Blackhole. section 7.1.1

A more elaborate comparison can parameterize account count and specify warmup, measurement, forks, and threads. Inspect these settings and the workload rather than trusting generated benchmark code or its assumptions automatically. section 7.1.2

Cursor code output part 1
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Benchmark)
@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@Threads(1)
public class AccountFinderJMHBenchmark {
    @Param({"1000", "10000", "100000"})
    private int accountCount;
    private AccountFinder accountFinder;
    private List<Account> accounts;
    private List<Integer> searchIds;
    private Random random;
    @Setup(Level.Trial)
    public void setupTrial() {
        random = new Random(42); #A
        accounts = generateAccounts(accountCount);
        accountFinder = new AccountFinder(accounts);
        searchIds = generateRandomIds(random, 100);
    }

See a generated benchmark configuration with three account counts and explicit warmup and measurement settings. section 7.1.2

File lookup versus caching

A word-existence benchmark compares the original file-based implementation with a cached implementation. The comparison helps evaluate a proposed optimization, but eager caching also introduces memory costs that lookup timings alone do not settle. section 7.4.1

Use JMH to compare isolated code paths, not as a substitute for end-to-end testing. Gatling can measure the whole system under request traffic, helping assess whether an isolated improvement matters at the endpoint level. section 7.2.3

How Vibe Engineering covers it

How deep it goes
Works through benchmark methods, configuration, and results, without explaining JMH installation.
Examples
Compares sequential and parallel account lookup, then file-based and cached word lookup.
What you can do after
Apply it via JMH to compare implementations with explicit workloads and weigh measured gains against operational costs.
How it connects
Connects microbenchmarking with JIT optimization, parallel streams, caching, and end-to-end performance testing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.