liveBook · Manning

Threshold

As explained in Fight Fraud with Machine Learning

A threshold is a cutoff value that determines which branch or class decision applies to a model output or feature value.

A threshold converts a model's fraud probability into a binary decision: output 1 when p > t and 0 when p <= t. Although 0.5 is a common default, the cutoff should reflect the application's precision and recall requirements. section 1.6.4

Use thresholds in progressively more deliberate ways:

  • Sweep candidate values from 0 to 1, calculate precision and recall for each, and use the precision-recall curve to select an appropriate trade-off. section 1.6.4
  • For a numerical rule, test distinct feature values, compare each resulting accuracy, and retain the best cutoff and comparison direction. section 2.2.2
  • For deployment, choose a model threshold that satisfies a required precision while maximizing recall. section 3.7.5

A phishing URL rule

Testing NumDash values identifies 2 as the optimal threshold for the rule NumDash < 2, producing 67.76% accuracy. Combining this rule with another numerical-feature rule through OR or AND changes accuracy to 69.74% or 75.19%. section 2.2.2

Figure 1.22 Precision-recall curve with hypothetical threshold values pointing to two different sets of precision and recall values.

A precision-recall curve makes threshold choices visible by linking candidate cutoffs to their resulting precision and recall. section 1.6.4

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains threshold sweeps, precision-recall selection, numerical-rule optimization, and deployment constraints.
Examples
Demonstrates a 0.5 probability cutoff, NumDash < 2, precision-recall curves, and a random-forest deployment threshold.
What it assumes
Assumes familiarity with model probabilities, binary labels, accuracy, precision, recall, and numerical feature comparisons.
What you can do after
Apply it via threshold sweeps and Python rules to select fraud decisions that meet accuracy or precision-recall requirements.
How it connects
Links probability, binary classification, precision, recall, precision-recall curves, numerical features, and random forests.

Also explained in

Compare how 20 books explain threshold →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.