Boruta selects features by comparing each feature’s model importance with the highest importance achieved by randomized shadow features. A feature earns a hit whenever it beats that shadow benchmark; a binomial significance test then supports retaining or rejecting it. section 6.2.2
Use Boruta when feature selection must identify all features that outperform random variation:
- Choose a decision-tree ensemble, such as random forests, gradient boosting, XGBoost, or LightGBM, as the estimator. section 6.2.2
- Transform the dataset first when using BorutaPy, because this implementation cannot work directly with pipelines. section 6.2.2
- Fit Boruta, inspect its support mask, and map selected positions back to the transformed feature names. section 6.2.2
Airbnb NYC feature selection
For an Airbnb NYC classification task, BorutaPy with an XGBoost tree estimator selected transformed variables including room type, neighbourhood, minimum nights, reviews, host-listing count, and availability. A LightGBM run produced a smaller selected set, illustrating that selected features depend on the tree ensemble used. section 6.2.2
Listing 6.10 Boruta selection
from XGBoost import XGBClassifier
from boruta import BorutaPy
xgb = XGBClassifier(booster='gbtree',
objective='reg:logistic',
n_estimators=300,
max_depth=4,
min_child_weight=3)
X = column_transform.fit_transform(data, target_median) #1
boruta_selector = BorutaPy(estimator=xgb, n_estimators='auto', verbose=2) #2
boruta_selector.fit(X, target_median) #3
selected_features = boruta_selector.support_ #4
selected_data = column_transform.get_feature_names_out()[selected_features]
print(selected_data)
Inspect the runnable BorutaPy workflow for transforming data, fitting an XGBoost estimator, reading the support mask, and printing selected feature names. section 6.2.2
