A piecewise linear gradient boosting tree combines decision-tree partitioning with linear models at the terminal nodes. The trees capture nonlinear structure, while each local linear model can extrapolate more effectively for unseen or outlying cases than an ordinary tree. section 5.5 section 5.2.2
Use this model when the prediction problem needs both local nonlinear behavior and extrapolation:
- Prefer it for time-series situations where future values may fall well outside the training targets. section 5.2.2
- Treat it as a compromise between gradient-boosted trees and linear models rather than as a replacement for either approach. section 5.5
- Expect ordinary gradient boosting to remain useful when extrapolation is not central to the task. section 5.2.2
Forecasting unusual demand
A supermarket-network forecasting problem may contain slow- and fast-moving goods, hierarchical series, lagged values, and moving averages. A piecewise linear boosting tree can model different local patterns while extending predictions toward demand values not represented in training data. section 5.2.2
Listing 5.16 LightGBM classifier
from lightgbm import LGBMClassifier
accuracy = make_scorer(accuracy_score)
cv = KFold(5, shuffle=True, random_state=0)
lgbm = LGBMClassifier(boosting_type='gbdt', #1
n_estimators=300,
max_depth=-1,
min_child_samples=3,
force_col_wise=True, #2
verbosity=0)
model_pipeline = Pipeline(
[('processing', column_transform),
('lightgbm', lgbm)]) #3
cv_scores = cross_validate(estimator=model_pipeline,
X=data,
y=target_median,
scoring=accuracy,
cv=cv,
return_train_score=True,
return_estimator=True) #4
mean_cv = np.mean(cv_scores['test_score'])
std_cv = np.std(cv_scores['test_score'])
fit_time = np.mean(cv_scores['fit_time'])
score_time = np.mean(cv_scores['score_time'])
print(f"CV Accuracy {mean_cv:0.3f} ({std_cv:0.3f})",
f"fit: {fit_time:0.2f}",
f"secs pred: {score_time:0.2f} secs") #5
This LightGBM pipeline shows the surrounding classifier workflow; setting boosting_type to gbdt and linear_tree to true selects piecewise linear terminal models. section 5.5
