Newton-raphson method
The Newton-Raphson method is an iterative numerical technique that uses first- and second-derivative information to find a root or optimize an objective.
Ensemble Methods for Machine Learning
Newton's method is an iterative optimization method that uses first- and second-derivative information to choose updates toward a local minimum.
- How it is used
- Motivates Newton boosting by showing how curvature-aware minimization trains models and approximates sequential ensemble updates.
- How deep it goes
- Derives gradient-and-Hessian updates, gives convergence pseudocode, and implements optimization for Branin and squared-loss classification.
- Examples
- Uses the Branin function, a saddle point, and a synthetic two-class squared-loss linear classifier.
- What you can do after
- Build it from scratch by computing gradients and Hessians, applying Newton updates, and stopping when parameter changes meet a tolerance.
Read how Ensemble Methods for Machine Learning explains newton-raphson method →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A Newton-Raphson update is an optimization step that uses first- and second-order derivatives to refine parameter values.
- How it is used
- Uses second-order optimization to update predictions within gradient boosting and XGBoost.
- How deep it goes
- Explains gradients, Hessians, curvature, adaptive learning rates, regularization, and a custom implementation, but avoids a full derivation.
- Examples
- Demonstrates a NewtonianGradientBoosting class that fits decision trees using gradient, Hessian, and regularization values.
- What you can do after
- Apply it via XGBoost or implement a Newton-style boosting update with Hessian-based regularization.
What to know first
- gradient boostingGradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient.
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
Related concepts
- Hessian matrixuses · A Hessian matrix is the square matrix of second-order partial derivatives of a scalar-valued function of multiple variables.
- gradient descentcontrasts with · Gradient descent is an iterative optimization method that updates parameters in the direction opposite an objective function’s gradient to reduce the objective’s value.
- Newton boostingused by · Newton boosting is a boosting approach that uses first- and second-order loss information to construct successive learners.
- second-order optimizationkind of · Second-order optimization is an optimization approach that uses first-derivative information together with curvature information, such as a Hessian or its approximation.
- LightGBMused by · LightGBM is an open-source gradient-boosting framework for efficiently training tree-based machine-learning models.
Questions about newton-raphson method
What is newton-raphson method?
The Newton-Raphson method is an iterative numerical technique that uses first- and second-derivative information to find a root or optimize an objective.
Which Manning books explain newton-raphson method?
Ensemble Methods for Machine Learning and Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI.
What should I know before learning newton-raphson method?
Manning books that teach newton-raphson method usually introduce gradient boosting and overfitting first.
What is the difference between newton-raphson method and gradient descent?
The Newton-Raphson method is an iterative numerical technique that uses first- and second-derivative information to find a root or optimize an objective. Gradient descent is an iterative optimization method that updates parameters in the direction opposite an objective function’s gradient to reduce the objective’s value.
What is newton-raphson method a kind of?
Newton-raphson method is a kind of descent method. A descent method is an iterative optimization technique that generates updates intended to reduce an objective function.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

