Binning groups numeric values into discrete, ordered bins and assigns each bin an integer while preserving the original order. The number of groups is commonly called k. section 5.4.3
Use binning when a continuous feature or target needs a more manageable representation:
- Accelerate decision-tree split evaluation by testing histogram-derived thresholds instead of every unique numeric value. section 5.4.3
- Convert a continuous target such as price into binary or multiclass classes by splitting at the mean or median, or by using deciles. section 4.2
- Turn a numeric feature into a categorical feature for association analysis, such as comparing deciles with Cramer's V. section 2.4.3
Listing 5.14 The histogram split
import numpy as np
def gini_impurity(y):
_, counts = np.unique(y, return_counts=True)
probs = counts / len(y)
return 1 - np.sum(probs**2) #1
def histogram_split(x, y, use_histogram, n_bins=256):
if use_histogram:
hist, thresholds = np.histogram(
x, bins=n_bins, density=False
) #2
else:
thresholds = np.unique(x) #3
best_score = -1
best_threshold = None #4
for threshold in thresholds: #5
left_mask = x <= threshold
right_mask = x > threshold
left_y = y[left_mask]
right_y = y[right_mask] #6
score = (
gini_impurity(left_y) * len(left_y)
+ gini_impurity(right_y) * len(right_y)
) #7
if score > best_score: q #8
best_threshold = threshold
best_score = score
return best_threshold, best_score #9
The implementation compares exact unique-value thresholds with histogram-based thresholds while calculating the best split using Gini impurity. section 5.4.3
Histogram splitting on latitude
For Airbnb latitude values, evaluating every unique threshold took about 46.9 seconds, while evaluating 256 histogram bins took about 562 milliseconds and produced the same reported split and score. section 5.4.3
