liveBook · Manning

F-score

As explained in Retrieval Augmented Generation, The Foundational Ideas

F1 score is the harmonic mean of precision and recall.

F-score, commonly reported as F1, is the harmonic mean of precision and recall. It balances the fraction of retrieved items that are relevant against the fraction of relevant items retrieved. section 14.1.1

Retrieval example

A financial assistant retrieves five documents, three of which are relevant, while four relevant documents exist in the corpus. Precision@5 is 0.6 and Recall@5 is 0.75, producing F1 = 2 × (0.6 × 0.75) / (0.6 + 0.75) = 0.667. section 14.1.1

F1 rewards balance rather than optimizing either component alone: a system with high precision but poor recall, or high recall but poor precision, receives a constrained score. In question answering, F1 can instead measure token overlap and give partial credit to answers that are close to or paraphrase the reference. section 5.3 section 14.1.1

Use F1 as a compact summary, but pair it with rank-aware measures when retrieval order affects generation.

  • Precision@k and Recall@k summarize how many retrieved documents are relevant and how many relevant documents were found. section 14.1.1
  • F1 does not show where relevant documents appear in the ranked list; MRR and NDCG@k capture rank position and graded usefulness more directly. section 14.1.1
  • For answer evaluation, F1 may reveal partial or paraphrased correctness even when Exact Match remains zero. section 5.3
Figure 14.1 Precision and recall. Of the items returned by the retriever (the selected set), precision measures the fraction that are relevant. Of all relevant items in the corpus, recall measures the fraction that the retriever surfaced.

Use the precision-and-recall diagram to see the retrieved set, relevant set, and overlap that F1 combines. section 14.1.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines the harmonic mean, works through retrieval numbers, and stops short of making rank position visible.
Examples
Uses a five-document financial compliance retrieval example and token-overlap scoring for question-answering responses.
What it assumes
Assumes precision and recall, while defining their retrieval meanings and calculating them from a concrete example.
What you can do after
Apply it to summarize precision-recall balance while checking rank-aware metrics for retrieval ordering.
How it connects
Connects precision and recall with Precision@k, Recall@k, MRR, NDCG@k, and Exact Match.

Also explained in

Compare how 20 books explain f-score →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.