Quadratic approximation

Q: How deeply does Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI cover quadratic approximation?

Defines second-order optimization and demonstrates gradient, Hessian, and regularized tree-target calculations without deriving the approximation.

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A second-order approximation estimates a function locally using both its first and second derivatives.

A quadratic approximation improves local optimization by using both first- and second-derivative information. In XGBoost, this second-order approximation is one of the techniques used for quicker gradient-boosting optimization. section 5.4.2

Newtonian boosting update

A Newtonian gradient-boosting implementation calculates a gradient and Hessian, then trains a decision tree on the adjusted target -gradient / (hessian + reg_lambda). With a constant Hessian and regularization value 0.25, the second-order information directly influences each prediction update.

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

liveBook manning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Related concepts

newton-raphson methoduses · The Newton-Raphson method is an iterative numerical technique that uses first- and second-derivative information to find a root or optimize an objective.
Taylor seriesuses · A Taylor series is a power series whose coefficients are determined by a function’s derivatives evaluated at a specified expansion point.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.