liveBook · Manning

Metric

As explained in Vibe Engineering

A metric is a quantitative measure used to assess a property or outcome of a software system or process.

A metric is a quantitative measure used to evaluate accuracy, performance, reliability, or system health. Useful metrics turn vague engineering claims into evidence, but excessive measurement can increase cognitive load and obscure the signal that matters. section 6.0 section 7.5

Use metrics as focused evidence rather than decoration:

  • Compare generated results with actual query results to measure model accuracy beyond a small set of integration-test cases. section 6.0
  • Instrument internal operations when black-box endpoint tests identify a bottleneck but cannot reveal which code path consumes time. section 7.3
  • Prefer targeted measurements and remove unnecessary instrumentation after optimization when measurement overhead or review scope becomes costly. section 7.5 section 7.3
  • Interpret benchmark numbers before changing production code, and check implicit trade-offs such as memory, startup time, and maintenance overhead. section 7.5

Separating endpoint performance

For the word-exists operation, separate timers measure file loading and scanning. With 1,200 invocations, loadFile reports p99 latency of 0.000730684 seconds, while scan reports 0.078602730 seconds, directing attention toward scanning rather than the whole endpoint. section 7.3

Listing 7.11 Measuring the word-exists logic
@Override
public boolean wordExists(String word) {
   Timer loadFile = metricRegistry.timer("loadFile");
   try (Scanner scanner = loadFile.time(() -> new
      Scanner(filePath.toFile()))) { #A
      
      Timer scan = metricRegistry.timer("scan");
      return scan.time( #B
          () -> {
            while (scanner.hasNextLine()) {
              String line = scanner.nextLine();
              if (word.equals(line)) {
                return true;
      }
      return false;
    });
  } catch (Exception e) {
      throw new RuntimeException("Problem in wordExists for word: " + word, e);
  }
}

Shows timer instrumentation around distinct stages of a Java method, including invocation timing and exception handling. section 7.3

How Vibe Engineering covers it

How deep it goes
Explains aggregate metrics and timer instrumentation down to percentiles, invocation counts, and measurement granularity, while cautioning against indiscriminate coverage.
Examples
Demonstrates query-accuracy comparisons, loadFile and scan timers, p99 latency, error rates, input-size distributions, and production latency spikes.
What it assumes
Assumes knowledge of interpreting measured results and teaches why black-box tests alone cannot establish model accuracy or internal performance causes.
What you can do after
Diagnose bottlenecks and evaluate changes using benchmarks, targeted timers, accuracy comparisons, and production aggregates.
How it connects
Connects metrics with benchmarking, integration testing, observability, logging, tracing, and performance optimization.

Also explained in

Compare how 20 books explain metric →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.