liveBook · Manning

Weighted average

As explained in Grokking Machine Learning, Second Edition

A weighted average is an average in which each value contributes according to an assigned weight.

A weighted average lets larger groups count more than smaller ones. When a dataset splits into leaves of different sizes, treating every leaf equally can hide how many samples each leaf represents. section 9.3

For a split of eight samples into groups of six and two, give the first group weight 6/8 and the second weight 2/8. Multiply each leaf’s Gini impurity or entropy by its share of the samples, then combine the results. This makes the larger leaf matter more. section 9.3

A six-and-two split

A split into datasets of sizes six and two uses weights 6/8 and 2/8. Weighting the leaves’ measures produces a Gini index of 0.333 and an entropy of 0.689, which can be compared with other possible splits. section 9.3

Figure 9.14 A split of a dataset of size eight into two datasets of sizes six and two. To calculate the average Gini index and the average entropy, we weight the index of the left dataset by 6/8 and that of the right dataset by 2/8. This results in a weighted Gini index of 0.333 and a weighted entropy of 0.689.

Compare the two leaf sizes, their fractional weights, and the resulting weighted Gini index and entropy. section 9.3

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Explains weighting procedurally through leaf proportions and resulting Gini index and entropy values without giving a general formula.
Examples
Uses an eight-sample dataset split into leaves of six and two samples.
What it assumes
Builds on prior use of Gini impurity and entropy for comparing decision-tree splits.
What you can do after
Evaluate/understand unequal-size decision-tree splits by weighting each leaf’s Gini impurity or entropy by its sample proportion.
How it connects
Connects weighted averages to decision trees, Gini impurity, entropy, accuracy, and information gain.

Also explained in

Compare how 6 books explain weighted average →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.