Extremely randomized trees (ERT), also called extra-trees in Scikit-learn, form an ensemble by randomly choosing the feature used at each tree split, then optimizing the split point for that feature. Compared with random forests, this creates more diverse, less-correlated trees and reduces prediction variance, while introducing more bias. section 5.1.3 section 5.6
The randomized split rule creates practical trade-offs:
- Choose ERT when high-dimensional, noisy, or imbalanced data make faster feature selection and robustness valuable. section 5.1.3
- Expect lower variance and faster training than random forests, but potentially higher bias because randomly selected features may reduce predictive accuracy. section 5.1.3 section 5.6
- Consider ERT especially for larger or wider datasets, where avoiding feature searches at every split saves time. section 5.1.3
Airbnb price classification
A practical test predicts whether an Airbnb NYC listing price is above or below the median. Replacing a random-forest model with ERT produced a slightly better result, while training was much faster with the same dataset and tree count. section 5.1.3
Listing 5.5 ERTs classifier
from sklearn.ensemble import ExtraTreesClassifier
accuracy = make_scorer(accuracy_score)
cv = KFold(5, shuffle=True, random_state=0)
model = ExtraTreesClassifier(n_estimators=300,
min_samples_leaf=3,
random_state=0) #1
column_transform = ColumnTransformer(
[('categories', categorical_onehot_encoding, low_card_categorical),
('numeric', numeric_passthrough, continuous)],
remainder='drop',
verbose_feature_names_out=False,
sparse_threshold=0.0) #2
model_pipeline = Pipeline(
[('processing', column_transform),
('modeling', model)]) #3
cv_scores = cross_validate(estimator=model_pipeline,
X=data,
y=target_median,
scoring=accuracy,
cv=cv,
return_train_score=True,
return_estimator=True) #4
mean_cv = np.mean(cv_scores['test_score'])
std_cv = np.std(cv_scores['test_score'])
fit_time = np.mean(cv_scores['fit_time'])
score_time = np.mean(cv_scores['score_time'])
print(f"{mean_cv:0.3f} ({std_cv:0.3f})",
f"fit: {fit_time:0.2f}",
f"secs pred: {score_time:0.2f} secs") #5
The Scikit-learn workflow combines ExtraTreesClassifier with one-hot encoding, numeric passthrough, a pipeline, and five-fold cross-validation. section 5.1.3
