An ensemble combines predictions from several independently trained models to produce a joint prediction. Different models may capture different aspects of the data, so pooling their perspectives can improve accuracy beyond a single model. section 18.1.2
Averaging classifier predictions
For four similarly capable classifiers, predict the validation inputs with each model and average their outputs: final_preds = 0.25 * (preds_a + preds_b + preds_c + preds_d). If the models differ substantially in quality, use learned validation weights instead, such as 0.5, 0.25, 0.1, and 0.15. section 18.1.2
Use ensembles with these practical considerations:
- Prefer diverse, well-performing models: each may make useful predictions for different reasons. section 18.1.2
- Average predictions when the component classifiers are more or less equally good. section 18.1.2
- Learn validation-data weights when some models are significantly better or worse than others. section 18.1.2
preds_a <- model_a |> predict(x_val) #1 preds_b <- model_b |> predict(x_val) preds_c <- model_c |> predict(x_val) preds_d <- model_d |> predict(x_val) final_preds <- 0.25 * (preds_a + preds_b + preds_c + preds_d) #2
Inspect a compact R example that obtains predictions from four models and averages them into final predictions. section 18.1.2
preds_a <- model_a |> predict(x_val) preds_b <- model_b |> predict(x_val) preds_c <- model_c |> predict(x_val) preds_d <- model_d |> predict(x_val) final_preds <- .5*preds_a + .25*preds_b + .1*preds_c + .15*preds_d #1
Compare equal averaging with a weighted combination that gives stronger models more influence. section 18.1.2
