An ensemble pools predictions from several independently trained models to produce a stronger prediction than one model alone. The models may capture different aspects of the data, so their complementary errors can yield a more accurate overall view. section 18.1.2
Average several classifiers
For classification, collect each classifier’s predictions at inference time and average them. Equal averaging is appropriate when the classifiers are roughly equally good; a much weaker model can reduce the ensemble’s quality. section 18.1.2
Practical rules of thumb:
- Prefer diverse, well-performing models: different architectures, assumptions, or random initializations can contribute different useful perspectives. section 18.1.2
- Replace equal averaging with a weighted average when model quality differs; learn the weights from validation data. section 18.1.2
- Treat ensembling as a practical performance-improvement technique, especially when pursuing competition-level results. section 18.0 section 20.4.1
preds_a = model_a.predict(x_val) #1 preds_b = model_b.predict(x_val) #1 preds_c = model_c.predict(x_val) #1 preds_d = model_d.predict(x_val) final_preds = 0.25 * (preds_a + preds_b + preds_c + preds_d) #2
The listing shows inference-time prediction pooling: four classifier outputs are added and divided equally to form the final prediction. section 18.1.2
