Hessian matrix
A Hessian matrix is the square matrix of second-order partial derivatives of a scalar-valued multivariable function, describing its local curvature.
Ensemble Methods for Machine Learning
A Hessian is the square matrix of second-order partial derivatives of a scalar-valued function.
- How it is used
- Guides Newton’s method and Newton boosting with second-order curvature information.
- How deep it goes
- Explains multivariate construction, inverse-Hessian updates, analytic and numerical derivatives, and leaf-node use without detailing large-problem handling.
- Examples
- Uses a three-variable function, the Branin function, focal-loss derivatives, and LightGBM training on breast-cancer data.
- What you can do after
- Apply it via LightGBM custom objectives or implement Newton updates from gradients and Hessians.
Read how Ensemble Methods for Machine Learning explains Hessian matrix →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
The square matrix of second-order partial derivatives of a scalar multivariable function, describing its local curvature.
- How it is used
- Uses second-order curvature information to adapt gradient-boosting updates and accelerate Newton-style optimization.
- How deep it goes
- Explains curvature, update behavior, regularization, and implementation, but defers detailed analytical and numerical derivations.
- Examples
- Demonstrates Hessian-weighted updates in a custom NewtonianGradientBoosting class and reports 0.82514 accuracy.
- What you can do after
- Apply it via a custom boosting implementation to scale gradient updates with curvature and regularization.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains Hessian matrix →
Related concepts
- newton-raphson methodused by · The Newton-Raphson method is an iterative numerical technique that uses first- and second-derivative information to find a root or optimize an objective.
- XGBoostused by · XGBoost is an open-source software library for efficient gradient-boosted decision-tree algorithms used in supervised machine learning.
- GPTQused by · GPTQ is a one-shot post-training weight-quantization method that uses calibration data and approximate second-order information to reduce precision while limiting accuracy loss.
- Jacobianequivalent to · The matrix of all first-order partial derivatives of a vector-valued function with respect to its inputs.
- Taylor seriesused by · A Taylor series is a power series whose coefficients are determined by a function’s derivatives evaluated at a specified expansion point.
- calibration dataproduced by · Calibration data is data used to measure, estimate, or adjust a model’s parameters, outputs, or behavior for a specified calibration objective.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.