liveBook · Manning

False positive

As explained in Vibe Engineering

A false positive is a classification that incorrectly reports a positive or high-risk condition when it is absent.

A false positive is a false alarm: a classifier reports a positive condition when it is absent. In a risk model, it means labeling a case high risk when it is not. False alarms can consume review time and erode trust, even when the system is fast and inexpensive. section 4.6.5 section 9.1

The decision threshold controls the tradeoff. Lowering it classifies more cases as high risk, potentially catching more real risks while increasing false positives. Raising it shifts more predictions to low risk, potentially missing real risks. Compare precision and recall across thresholds rather than choosing a cutoff on accuracy alone. section 4.6.5 section 4.6.3

A threshold changes the classification

At a threshold of 0.3, a predicted probability of 0.31 is classified as high risk. At a threshold of 0.6, a probability of 0.55 is classified as low risk. Whether either classification is a false positive depends on the actual condition. section 4.6.5

Figure 4.11 Precision vs. Recall at Different Thresholds.

Use the precision and recall plot to compare the effects of different thresholds. section 4.6.5

Review noise becomes a hidden cost

A CI code-review agent switched to a cheaper model. Its false-positive review comments rose from roughly 8% to 18% as it flagged correct code as problematic. Developers began ignoring its comments, and senior engineers returned to full manual reviews. section 9.1

Check false alarms against the cost of missed problems and the effort needed to investigate findings. Generated tests also need review: their validity and design should be checked so they do not introduce false positives. section 9.1 section 2.4.1

How Vibe Engineering covers it

How deep it goes
Explains threshold tradeoffs with concrete classifications and a precision-recall plot, and describes the operational cost of noisy review findings.
Examples
Illustrates false positives with medication-risk predictions and a CI reviewer that flags correct code.
What you can do after
Evaluate classification thresholds and review findings by weighing false alarms against missed risks and investigation costs.
How it connects
Connects false positives to classification thresholds, precision, recall, false negatives, and automated code review.

Also explained in

Compare how 26 books explain false positive →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.