liveBook · Manning

One-vs-rest strategy

As explained in Grokking Statistics

A multiclass classification strategy that trains one binary classifier per category against all other categories.

One-vs-rest is a way to turn a multiclass classification problem into several binary ones. For each category, train a classifier that asks, “Is it this category or not?” section 8.3

For a new example:

  • Run it through every category-specific binary classifier. section 8.3
  • Compare the probabilities returned by the classifiers. section 8.3
  • Choose the category with the highest probability. section 8.3

Sorting animal photos

Classify each photo with three questions: “Is it a dog or not?”, “Is it a cat or not?”, and “Is it a bird or not?” The strongest probability determines the label. section 8.3

Logistic regression can use this strategy to handle multiple categories, while multinomial logistic regression calculates all category probabilities simultaneously under the hood. section 8.3

How Grokking Statistics covers it

How deep it goes
Defines the training and prediction pattern with an example, but does not develop its implementation or internal mathematics.
Examples
Uses dog-versus-rest, cat-versus-rest, and bird-versus-rest classifiers to explain the strategy.
What it assumes
Builds from binary outcomes and logistic regression before extending them to multiple categories.
What you can do after
Evaluate how multiple binary predictions can produce one multiclass label.
How it connects
Connects binary classification, logistic regression, and multinomial logistic regression.
Cover of Grokking Statistics
liveBookmanning.com
Read Grokking Statistics on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Statistics. The section links point to the parts of the book it draws on.