liveBook · Manning

F-score

As explained in Vibe Engineering

The F1 score is the harmonic mean of precision and recall.

A default classification threshold of 0.5 may not provide the best balance between precision and recall. The F1 score, their harmonic mean, offers a criterion for choosing a threshold instead of guessing. section 4.6.5

Selecting a threshold

For a medical-text model, the best reported F1 score was 0.9559 at a threshold of 0.5999. Five-fold cross-validation produced an average accuracy of 0.9543, with similar results across folds. Even so, the high F1 score warrants a sanity check for potential data leakage; near-perfect scores on a small dataset can also indicate overfitting. section 4.6.5

Figure 4.11 Precision vs. Recall at Different Thresholds.

Compare precision and recall across thresholds before selecting the threshold with the best F1 score. section 4.6.5

How Vibe Engineering covers it

How deep it goes
Defines F1 as a harmonic mean and applies it to threshold selection, with implementation delegated to a linked helper.
Examples
Reports a medical-text model’s best F1 of 0.9559 at a threshold of 0.5999.
What you can do after
Evaluate model thresholds using F1 while checking high scores for potential data leakage.
How it connects
Connects F1 to precision, recall, cross-validation, overfitting, and data leakage.

Also explained in

Compare how 20 books explain f-score →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.