Weighted majority vote

As explained in Grokking Machine Learning

A prediction aggregation rule in which each learner's vote contributes according to an assigned weight.

A weighted majority vote combines predictions from several learners, giving each learner influence according to its score. In AdaBoost, scores come from learner accuracy: accurate learners get positive scores, chance-level learners get zero, and less accurate learners get negative scores. section 12.3

To make a binary prediction:

• Represent each learner's prediction as +1 or −1, then multiply it by that learner's score. section 12.3
• Add the weighted votes; predict class 1 when the sum is at least zero, and class 0 otherwise. section 12.3

Three friends giving advice

A highly reliable friend gets a large positive score, an unpredictable friend gets a score near zero, and a friend who is usually wrong gets a large negative score. Their weighted advice combines into one prediction. section 12.3

Figure 12.14 How to combine the weak learners into a strong learner in the AdaBoost model. We score each of the weak learners using the log-odds and make them vote based on their scores (the larger the score, the more voting power that particular learner has). Each of the regions in the bottom diagram has the sum of the scores of the weak learners. Note that to simplify our calculations, the predictions from the weak learners are +1 and –1, instead of 1 and 0.

See how learner scores determine voting power and combine across regions. section 12.3

How Grokking Machine Learning covers it

How deep it goes

Explains accuracy-based log-odds scores and combines signed votes to produce binary predictions.

Examples

Uses a three-friend analogy, a triangle-and-square dataset, and spam classification with six depth-one trees.

What you can do after

Apply Scikit-Learn's AdaBoostClassifier to fit and score a spam classifier using lottery and spam word counts.

How it connects

Connects weighted voting to AdaBoost, boosting, weak learners, log-odds, and ensemble learning.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

• boostingused by · Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.