A similarity score measures how alike the elements in a set are. For values a₁ through aₙ, square their sum and divide by the number of elements; a denominator parameter can optionally provide regularization. section 12.5
Compare three sets
The sets {10, −10, 4}, {7, 7, 7}, and {7} illustrate the measure: the repeated sevens are most similar, while the mixed values are least similar. section 12.5
In XGBoost, the score helps build decision trees by comparing possible splits. A split is preferred when its leaves have a higher combined score; similarity gain then helps decide whether to keep the split during pruning. section 12.5
Figure 12.23 The second weak learner in our XGBoost classifier. For each of the nodes, we can see the split based on the age feature, the labels corresponding to that node, and the similarity score for each set of labels. The split chosen for each node is the one that maximizes the combined similarity score of the leaves. For each of the leaves, you can see the corresponding labels and their similarity score.
See how a split based on age groups labels into leaves and compares their combined similarity score. section 12.5
