An objective function defines what model training should optimize: it quantifies prediction error and supplies the direction for improving estimates. In gradient boosting, each successive tree learns corrections from gradients, moving predictions toward the target. section 5.4.1 section 5.2.1
Choose the objective according to the prediction problem:
- Use reg objectives for regression, binary objectives for binary classification, and multi objectives for multiclass classification. section 5.4.1
- Use count for discrete events, survival for time-to-event analysis with censoring, and rank for ranking problems. section 5.4.1
- Use reg:gamma or reg:tweedie when modeling distributions associated with insurance claim amounts; Poisson supports event-frequency modeling. section 5.4.1
For binary classification, reg:logistic and binary:logistic are comparable to log-loss. The objective differs from an evaluation metric: the objective drives optimization, whereas evaluation metrics assess results. section 5.4.1
Gradient boosting classifier
A compact GradientBoosting implementation computes the gradient as y_pred - y_true, initializes predictions from the target mean, and repeatedly fits trees to corrections. On Airbnb NYC data, a train/test workflow with 300 trees, learning rate 0.1, and maximum depth four reports 0.82503 accuracy.
