How deeply does Machine Learning System Design cover specificity?

Defines specificity through negative-case counts and compares its class-balance behavior with precision, without developing a deeper derivation.

Specificity

As explained in Machine Learning System Design

The proportion of actual negative cases that a classifier correctly identifies.

In fraud detection, most transactions may be genuine. Specificity, also called the true negative rate, is the share of genuine transactions correctly left unflagged: TN / (TN + FP). Unlike precision, it does not change solely because the balance of fraud and genuine cases changes. section 5.2

A low false-positive rate can still mean many bad alerts

Suppose two models each catch 80 of 100 fraud cases among 100,000 genuine transactions. Model A falsely flags 20 genuine transactions; model B flags 920. Their specificities are 99.98% and 99.08%, respectively. Both look high, but only 80 of model B’s 1,000 fraud alerts are correct. Check precision as well when the usefulness of alerts matters. section 5.2

How Machine Learning System Design covers it

liveBook manning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Starting Data Analytics with Generative AI and PythonSpecificity is the proportion of actual negative instances that a classification method correctly identifies.
liveBook manning.com

Grokking Machine LearningA classification metric equal to the proportion of actual negative examples correctly identified, also called the true negative rate.
liveBook manning.com

Statistics Slam DunkSpecificity is the proportion of actual negative cases that a classification procedure correctly identifies.
liveBook manning.com

Statistics Every Programmer NeedsSpecificity is the proportion of actual negative instances that a classifier correctly identifies as negative.
liveBook manning.com

Fabulous Adventures in Data Structures and AlgorithmsTest specificity is the conditional probability that a medical test is negative when the tested condition is absent.

Compare how 6 books explain specificity →

Related concepts

false-positive ratecontrasts with · The false-positive rate is the proportion of actual negative cases that are incorrectly classified or identified as positive.
precisioncontrasts with · Precision is the proportion of items identified as positive, selected, or retrieved that satisfy the relevant correctness or relevance criterion.
recallcontrasts with · Recall is the fraction of relevant, positive, or target outcomes that a system successfully identifies, retrieves, or captures.
receiver operating characteristicused by · A receiver operating characteristic curve plots a binary classifier’s true-positive rate against its false-positive rate as the decision threshold varies.
thresholdcontrasts with · A threshold is a boundary value against which a quantity is compared to determine whether a condition is met or an action or decision is triggered.
FinBERT-PT-BRmeasures · FinBERT-PT-BR is a Portuguese sentiment-classification model trained on financial news.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.