A residual is the difference between a target label and a model’s prediction: label − prediction. In boosting, each new tree is trained to fit the remaining residuals, helping correct earlier predictions. section 12.4 section 12.5
App engagement
If a model predicts 4 engagement days for every user and one user’s label is 7, the residual is 7 − 4 = 3. After a tree adjusts the predictions, residuals are recalculated from the new combined predictions. section 12.4 section 12.5
Table 12.4 The predictions from the first weak learner are the average of the labels. The second weak learner is trained to fit the residuals of the first weak learner. (view table figure)
| Feature (age) |
Label (engagement) |
Prediction from weak learner 1 |
Residual |
Prediction from weak learner 2 |
| 10 |
7 |
4 |
3 |
3 |
| 20 |
5 |
4 |
2 |
2 |
| 30 |
7 |
4 |
3 |
2 |
| 40 |
1 |
4 |
–3 |
–2.667 |
| 50 |
2 |
4 |
–2 |
–2.667 |
| 60 |
1 |
4 |
–3 |
–2.667 |
| 70 |
5 |
4 |
1 |
0.5 |
| 80 |
4 |
4 |
0 |
0.5 |
Compare each user’s engagement label with the first tree’s prediction to see the residuals fitted by the next tree. section 12.4
