liveBook · Manning

Confusion matrix

As explained in Natural Language Processing in Action, Second Edition

A confusion matrix is a table counting predictions by their actual and predicted classes.

A confusion matrix counts predictions by actual and predicted class, making classification mistakes visible instead of hiding them inside one accuracy score. For a toxic-comment classifier, it separates correctly and incorrectly identified toxic and nontoxic comments. section 4.2.1

Read the two error types in relation to the actual labels:

  • False negative: an actually toxic comment predicted as nontoxic. section 4.2.1
  • False positive: an actually nontoxic comment predicted as toxic. section 4.2.1
  • Recall, or sensitivity, measures how many instances of the class of interest the model identifies correctly. section 4.2.1

Inspecting a scikit-learn classifier

With y_test and predictions from a held-out set, confusion_matrix(y_test, lda.predict(X_test)) returns [[1261, 913], [201, 125]]. ConfusionMatrixDisplay.from_estimator(...) turns those counts into a labeled plot for easier diagnosis.

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Explains error categories, visual interpretation, and recall through a worked binary-classification result.
Examples
Uses toxic-comment predictions, a numeric matrix, and a scikit-learn Matplotlib display.
What it assumes
Assumes binary labels, held-out predictions, and basic scikit-learn evaluation while defining false-positive and false-negative terminology.
What you can do after
Diagnose model mistakes with scikit-learn counts and visual displays, then interpret recall for the toxic class.
How it connects
Connects classification accuracy with false positives, false negatives, recall, sensitivity, and train/test evaluation.
Cover of Natural Language Processing in Action, Second Edition
liveBookmanning.com
Read Natural Language Processing in Action, Second Edition on liveBook Get the book at manning.com

Also explained in

Compare how 14 books explain confusion matrix →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.