A boosted decision tree is an ensemble that trains decision trees sequentially. Each new tree corrects earlier errors, and gradient descent combines their contributions into a prediction. Unlike random forest, which averages trees trained in parallel, boosting builds an ordered sequence of corrective models. In fraud detection, this approach often outperforms random forest, although results depend on the dataset and task. section 3.5 section 3.9
A practical boosted-tree workflow includes:
- Train an XGBoost classifier with randomized hyperparameter search and three-fold cross-validation. section 3.5.1
- Tune settings such as tree depth, number of estimators, and scale_pos_weight for imbalanced fraud labels. section 3.5.1 section 3.9
- Compare the best boosted-tree model with the best random forest using the same train-test split and evaluation process. section 3.5 section 3.5.1
- Inspect feature importance and plot individual trees to understand which inputs and thresholds influence predictions. section 3.5.3
Tracing a fraud prediction
A plotted tree can compare a transaction's amount with a threshold such as 2041690.25, route the record through conditional branches, and end at a fraudulent or genuine leaf. The complete model passes the transaction through all four trees in order. Each tree contributes a small score or log-odds adjustment; the combined score is typically passed through a logistic function to produce the final fraud probability. section 3.5.3
xgb = XGBClassifier(use_label_encoder=False, eval_metric='logloss', random_state=42) #A
param_dist = {'max_depth': randint(1, 5), 'n_estimators': randint(2, 10), 'scale_pos_weight': [1, 2, 5, 10] # XGBoost alternative to class_weight
} #B
model_xgb = train_model(xgb, param_dist) #C
Inspect the XGBoost training listing, including classifier setup, randomized hyperparameters, class weighting, and the training call. section 3.5.1







