In AdaBoost, a learner score sets how strongly a weak learner’s prediction counts in the ensemble’s weighted vote. The score is based on accuracy using log-odds: it is positive above 50% accuracy, zero at 50%, and negative below 50%. section 12.3
Three friends as classifiers
A usually truthful friend gets a high positive score; a usually lying friend gets a high negative score; and an unpredictable friend gets a score near zero. A negative score lets a learner that performs below chance contribute in the opposite direction. section 12.3
Table 12.2 Several values for the accuracy of a weak classifier, with the corresponding score, calculated using the log-odds. Notice that the models with very low accuracy get large negative scores, the values with very high accuracy get large positive scores, and the values with accuracy close to 0.5 get scores close to 0. (view table figure)
| Accuracy |
Log-odds (score of the weak learner) |
| 0.01 |
–4.595 |
| 0.1 |
–2.197 |
| 0.2 |
–1.386 |
| 0.5 |
0 |
| 0.8 |
1.386 |
| 0.9 |
2.197 |
| 0.99 |
4.595 |
Compare example accuracies with their log-odds scores: accuracy near 50% gives a score near zero, while accuracy near 0% or 100% gives a large-magnitude score. section 12.3
