liveBook · Manning

Label

As explained in Fight Fraud with Machine Learning

A ground-truth label is a reference annotation assigned to an instance according to the accepted or manually verified content or outcome.

Ground truth is the reference label assigned to an instance for comparison with a model’s output. In fraud detection, it commonly records the accepted outcome of a manual review or dataset annotation: fraudulent versus genuine, illicit versus licit, real versus fake, or same person versus different people. A prediction is not ground truth; it is the system’s estimate of that reference outcome. section 1.6 section 9.2.3 section 8.2.2

Email-phishing evaluation

For 100 emails, manual reviewers identify 10 as fraudulent and assign those results as ground-truth labels. A detector flags 15 emails as suspicious. Comparing its flags with the reviewer labels reveals correct detections, missed fraud, and genuine emails wrongly flagged as fraud. section 1.6 section 1.6.2

Ground truth can take several forms, depending on the task:

  • A binary class label, such as CLASS_LABEL = 0 for a genuine webpage and 1 for a fraudulent webpage. section 2.1
  • An actual transaction-fraud outcome, distinct from a rule-based proxy label such as isFlaggedFraud. section 3.3.1
  • A pairwise identity label: 1 when two face images belong to one person and 0 when they belong to different people. section 7.2.2
  • Spatial annotations: receipt forgery regions expressed as bounding boxes and converted into pixel-level segmentation masks. section 6.1 section 6.3.1

Use ground truth at two points in the workflow. First, inspect its distribution before training: fraud labels may be extremely rare, so class imbalance affects how training is handled. Then compare predictions with held-out labels during evaluation. Loss functions use prediction-label differences to tune parameters during training; evaluation can binarize predictions and measure accuracy and F1, with F1 helping when classes are imbalanced. section 3.3.2 section 9.2.3 section 8.2.2

Figure 1.19 An example showing how to calculate the accuracy of a hypothetical email phishing fraud detection system.

This phishing example makes the distinction concrete: reviewer-assigned outcomes supply the reference labels against which binary fraud predictions are evaluated. section 1.6

How Fight Fraud with Machine Learning covers it

How deep it goes
Defines reviewer-assigned labels and demonstrates their use in loss calculation, confusion matrices, and evaluation metrics.
Examples
Uses reviewed phishing emails, webpage fraud classes, transactions, receipt bounding boxes, face pairs, Bitcoin transactions, and voice samples.
What it assumes
Introduces binary reviewer labels directly while later applying them in PyTorch training workflows.
What you can do after
Evaluate/understand model outputs by comparing predictions with manually reviewed or annotated reference labels.
How it connects
Connects ground truth to classification, confusion matrices, class imbalance, binary cross-entropy, F1 score, and segmentation.

Also explained in

Compare how 29 books explain label →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.