liveBook · Manning

Error rate

As explained in Timeless Algorithms, The Foundational Ideas

An error rate is the proportion of predictions or classifications that are incorrect.

An error rate is the proportion of predictions or classifications that are incorrect. In a classification forest, it can measure how often predictions disagree with known classes. section 9.5

For out-of-bag evaluation, each observation is predicted by trees that left it out of their bootstrap samples. The proportion of these predictions that misclassify their observations is the forest’s out-of-bag error rate. This provides internal validation without setting aside a separate validation sample. section 9.5

Out-of-bag classification

With 500 trees, an observation is out of bag for about 184 trees on average. Those trees vote on its class; if their majority prediction differs from its known class, that observation contributes an error. section 9.5

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines classification error as a proportion and explains its calculation from out-of-bag predictions.
Examples
Uses majority-vote out-of-bag predictions and compares them with known classes; contrasts classification error with regression error measures.
What it assumes
Assumes known outcomes for comparison and explains how trees that omit an observation can provide its prediction.
What you can do after
Evaluate a classification forest internally by comparing out-of-bag predictions with known classes.
How it connects
Connects error rates to random forests, bootstrap sampling, confusion matrices, cross-validation, and hypothesis testing.

Also explained in

Compare how 10 books explain error rate →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.