liveBook · Manning

Gradient descent

As explained in Fight Fraud with Machine Learning

Gradient descent is an iterative optimization method that updates parameters in the direction that reduces an objective function.

Gradient descent trains a neural network by iteratively adjusting its weights to reduce a defined loss function. For binary classification, binary cross entropy compares predicted probabilities with labeled outputs. section 4.1.1

A training pass follows a concrete sequence:

  • Initialize network weights with random values. section 4.1.1
  • Run a forward pass to produce predictions and calculate loss against the actual labels. section 4.1.1
  • Backpropagate the loss through the network, using the chain rule to calculate each weight's contribution. section 4.1.1
  • Update the weights using the calculated gradients, repeating the process to minimize loss. section 4.1.1 section 4.4

Dog-versus-cat classification

A neural network receives labeled dog and cat images, predicts the probability that each image contains a dog, calculates binary cross entropy, and updates its weights through backpropagation and gradient descent. section 4.1.1

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains initialization, forward passes, loss, chain-rule backpropagation, and updates, while deferring detailed optimizer mechanics.
Examples
Uses a dog-versus-cat image classifier, binary cross entropy, network weights, and a gradient-descent training diagram.
What it assumes
Assumes labeled training data and a defined loss function, while introducing weight initialization, forward passes, gradients, and backpropagation.
What you can do after
Apply it to understand and train a neural network by calculating loss, propagating gradients, and updating weights.
How it connects
Connects gradient descent with backpropagation, loss functions, binary cross entropy, neural networks, SGD, Adagrad, and Adam.

Also explained in

Compare how 24 books explain gradient descent →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.