How deeply does Ensemble Methods for Machine Learning cover marginalization?

Defines summing out, gives an empirical prediction-and-averaging procedure, and notes computational cost without developing formal integration.

Marginalization

As explained in Ensemble Methods for Machine Learning

Marginalization is the computation of a distribution or expectation for selected variables by summing or integrating over the others.

Marginalization explains how a model’s output relates to one selected feature by averaging away the influence of all remaining features. In a partial dependence plot (PDP), hold the selected feature at a chosen value, predict across the examples, and average those predictions. Repeating this for multiple values reveals the feature–target relationship. section 9.3.2

Compute a marginal effect with this practical procedure:

Choose the feature and a candidate value a. section 9.3.2
Replace that feature’s entire data column with a, leaving the other features unchanged. section 9.3.2
Run the black-box model on the modified data and average the resulting prediction vector. section 9.3.2
Repeat for other values to produce the full partial dependence relationship; approximation avoids an infeasible exact integral. section 9.3.2

Two points on a partial dependence plot

For the third feature, create modified datasets with the feature fixed at two chosen values, generate predictions for each dataset, and average each prediction set to obtain two PDP points.

How Ensemble Methods for Machine Learning covers it

Cover of Ensemble Methods for Machine Learning

liveBook manning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

partial dependence plotuses · A partial dependence plot shows a model's average predicted response as selected features vary, averaging over the values of the remaining features.
retrieval-augmented generationimplements · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
marginal probabilityproduces · A marginal probability or distribution describes a variable or event without conditioning on other variables, obtained from a joint distribution by summing or integrating over them.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.