liveBook · Manning

Pseudo-Huber loss

As explained in Ensemble Methods for Machine Learning

Pseudo-Huber loss is a smooth approximation to Huber loss that transitions continuously between quadratic and approximately absolute-error behavior.

Pseudo-Huber loss gives Newton boosting a smooth alternative to Huber loss. It preserves Huber’s outlier-limiting behavior while remaining differentiable enough for second-derivative methods. section 7.3.1 section 7.3.2

Its behavior follows the residual size:

  • For residuals close to zero, it behaves like squared error, producing 1/2·(y − f(x))². section 7.3.1
  • For larger residuals, it approximates Huber loss, which transitions toward scaled absolute error and limits outlier influence. section 7.3.1
  • Unlike Huber and absolute error losses, it supports Newton boosting because it is differentiable for second-derivative calculations. section 7.3.2 section 7.4.4

Using pseudo-Huber loss with XGBoost

Configure XGBRegressor with objective='reg:pseudohubererror', then tune tree depth, learning rate, feature sampling, and L1/L2 regularization through randomized cross-validation. In one regression comparison, the resulting model reached test MSE 0.0947 and test MAE 0.2244, close to LightGBM’s Huber result. section 7.3.2

How Ensemble Methods for Machine Learning covers it

How deep it goes
Explains residual behavior and differentiability, then demonstrates configuration and tuning; it stops short of deriving the full formula.
Examples
Uses XGBRegressor configuration, randomized cross-validation, evaluation metrics, and a bike-rental ensemble comparison.
What it assumes
Assumes regression, residuals, loss functions, gradient boosting, and second derivatives; reinforces why smoothness matters for Newton boosting.
What you can do after
Apply it via XGBoost by selecting reg:pseudohubererror and tuning ensemble, regularization, and tree-learning parameters.
How it connects
Connects pseudo-Huber loss to Huber loss, squared error, absolute error, Newton boosting, gradient boosting, and XGBoost.
Cover of Ensemble Methods for Machine Learning
liveBookmanning.com
Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.