A weighted average combines several values while giving each one a chosen influence. In classifier ensembling, it pools model predictions instead of trusting a single model. section 18.1.2
Equal versus learned influence
Four classifier outputs can receive equal influence by multiplying their combined predictions by 0.25. A smarter ensemble might give weights of 0.5, 0.25, 0.1, and 0.15, usually favoring stronger classifiers. section 18.1.2
Use these practical rules:
- Learn ensemble weights on validation data rather than assuming every classifier is equally good. section 18.1.2
- Search for weights with random search or the Nelder–Mead optimization algorithm. section 18.1.2
- Prioritize model diversity: differently biased classifiers can cancel one another’s errors, while similarly biased models preserve the same weakness. section 18.1.2
preds_a = model_a.predict(x_val) #1 preds_b = model_b.predict(x_val) #1 preds_c = model_c.predict(x_val) #1 preds_d = model_d.predict(x_val) final_preds = 0.25 * (preds_a + preds_b + preds_c + preds_d) #2
Shows the equal-weight implementation for combining four validation predictions. section 18.1.2
preds_a = model_a.predict(x_val) preds_b = model_b.predict(x_val) preds_c = model_c.predict(x_val) preds_d = model_d.predict(x_val) final_preds = 0.5 * preds_a + 0.25 * preds_b + 0.1 * preds_c + 0.15 * preds_d#1
Shows unequal weights that assign different influence to four classifier predictions. section 18.1.2
