When explaining one prediction, SHAP builds surrogate examples by keeping coalition features from the example of interest and replacing other features with values from another random training example. The Shapley kernel weights each surrogate according to coalition size rather than distance: πₓ(z) = ((d − n_z − 1)! n_z!) / d!, where d is the total number of features and n_z is the number included in the coalition. This gives comparable weight to coalitions with similar numbers of included and excluded features. section 9.4.2
KernelSHAP uses the weighting in a practical sequence:
- Construct coalition-based surrogate examples in the locality of the prediction to explain. section 9.4.2
- Weight those examples with the Shapley kernel and fit a linear regression model. section 9.4.2
- Interpret the fitted linear-model weights as approximate Shapley values for the features. section 9.4.2
Figure 9.16 All possible coalitions of Ava, Ben, Cam, and Dev, and their corresponding values (in units of $1,000). The last coalition contains all four friends and has a value of $20,000, the total prize money. There is one coalition of size 0, four coalitions of size 1, six coalitions of size 2, four coalitions of size 3, and one coalition of size 4. This table is called the characteristic function.
A coalition table makes the weighting intuition concrete: feature subsets range from the empty coalition through the complete coalition, with different numbers of coalitions at each size. section 9.4.2
