XGBClassifier applies XGBoost to supervised classification, making it a practical choice for tabular data such as Airbnb listings. It combines boosted decision trees with regularization, sparse-data handling, parallel construction, and optional distributed or out-of-core computation. section 5.4
A practical configuration exposes the main training trade-offs:
- Set n_estimators to control the number of boosted trees; the demonstrated classifier uses 300. section 5.4
- Limit max_depth and require a minimum child weight to constrain tree complexity; the example uses four levels and three examples. section 5.4
- Use early_stopping_rounds, eval_metric, and eval_set to halt training when validation performance stops improving. section 5.4.4
Cross-validated tabular classification
Place XGBClassifier inside a preprocessing-and-model pipeline, then evaluate it with shuffled five-fold cross-validation. The demonstrated configuration uses the gbtree booster, 300 estimators, depth four, and minimum child weight three. section 5.4
Listing 5.12 XGBoost classifier
from XGBoost import XGBClassifier
accuracy = make_scorer(accuracy_score)
cv = KFold(5, shuffle=True, random_state=0)
xgb = XGBClassifier(booster='gbtree', #1
objective='reg:logistic', #2
n_estimators=300,
max_depth=4,
min_child_weight=3) #3
model_pipeline = Pipeline(
[('processing', column_transform),
('XGBoost', xgb)])
cv_scores = cross_validate(estimator=model_pipeline,
X=data,
y=target_median,
scoring=accuracy,
cv=cv,
return_train_score=True,
return_estimator=True)
mean_cv = np.mean(cv_scores['test_score'])
std_cv = np.std(cv_scores['test_score'])
fit_time = np.mean(cv_scores['fit_time'])
score_time = np.mean(cv_scores['score_time'])
print(f"{mean_cv:0.3f} ({std_cv:0.3f})",
f"fit: {fit_time:0.2f}",
f"secs pred: {score_time:0.2f} secs") #4
Browse the complete XGBClassifier pipeline listing, including preprocessing, cross-validation, scoring, and reported fit and prediction times. section 5.4
Listing 5.15 Applying early stopping to XGBoost
train, test = train_test_split(
range(len(data)),
test_size=0.2,
random_state=0
) #1
train, validation = train_test_split(
train,
test_size=0.2,
random_state=0
) #2
xgb = XGBClassifier(booster='gbtree',
objective='reg:logistic',
n_estimators=1000,
max_depth=4,
min_child_weight=3,
early_stopping_rounds=100, #3
eval_metric='error') #4
X = column_transform.fit_transform(data.iloc[train])
y = target_median[train]
Xv = column_transform.transform(data.iloc[validation])
yv = target_median[validation]
xgb.fit(X, y, eval_set=[(Xv, yv)], verbose=False) #5
Xt = column_transform.transform(data.iloc[test])
yt = target_median[test]
preds = xgb.predict(Xt)
score = accuracy_score(y_true=yt, y_pred=preds)
print(f"Accuracy: {score:0.5f}") #6
Browse the early-stopping workflow, which separates training, validation, and test data and passes the validation set to fit. section 5.4.4
