An average latency can look acceptable while slow requests frustrate users. A percentile describes a threshold in the distribution: p99 latency is the value at or below which approximately 99% of observed latencies fall. It is a tail-latency measure, not the maximum or the average. section 7.2.3
Use latency percentiles to assess responsiveness and service-level targets, but include traffic frequency when deciding what to optimize. For an estimate of aggregate processing demand, multiply mean execution time by request rate—not p99 by request rate. section 7.2.3 section 7.2
Find the slow phase
A benchmark reports p99 latency of almost 82 ms for word validation, which handles about 95% of requests, versus 33 ms for the less frequent word-of-the-day endpoint. Separate timers then report p99 values of about 0.7 ms for loading and 78.6 ms for scanning. The phase-level measurements point toward scanning rather than file loading as the immediate bottleneck. section 7.2.3 section 7.3
Listing 7.11 Measuring the word-exists logic
@Override
public boolean wordExists(String word) {
Timer loadFile = metricRegistry.timer("loadFile");
try (Scanner scanner = loadFile.time(() -> new
Scanner(filePath.toFile()))) { #A
Timer scan = metricRegistry.timer("scan");
return scan.time( #B
() -> {
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
if (word.equals(line)) {
return true;
}
return false;
});
} catch (Exception e) {
throw new RuntimeException("Problem in wordExists for word: " + word, e);
}
}Separate timers expose the loading and scanning phases so their latency distributions can be compared. section 7.3
Instrumentation has a cost: measuring every path can affect performance, so choose granularity carefully. A dashboard full of aggregates and averages may also make a bottleneck harder to diagnose than direct phase-level percentiles. Verify a proposed optimization against measurements rather than relying on a plausible code-level explanation alone. section 7.3 section 7.3.1
Percentiles can summarize other distributions too. For SQL similarity scores, p50 describes typical overlap, while p90 and p99 expose stronger predictions. For cache eviction, request intervals can inform an initial timeout; monitor hits and misses afterward, since the observed distribution and cache behavior may change. section 6.4.1 section 7.4












