liveBook · Manning

Adam

As explained in Fight Fraud with Machine Learning

Adam is an adaptive stochastic optimization algorithm that updates model parameters using estimates of first and second moments of gradients.

Adam is an adaptive stochastic optimizer for neural networks. It improves on stochastic gradient descent by assigning different learning rates to weights and incorporating a running average of past gradients, or momentum, into updates. section 4.1.1

Use Adam after defining the model and loss function, then update it inside the training loop: clear gradients, run a forward pass, calculate loss, backpropagate, and call the optimizer step. Learning rate and weight decay remain tunable choices. section 8.2.2 section 9.2.3

Classification training with Adam

A fake-voice AudioTransformer uses Adam with a learning rate of 0.001 and binary cross entropy during ten training epochs. An MLP for licit-versus-illicit Bitcoin transactions uses Adam with learning rate 0.01 and weight decay of 1e-4, then evaluates accuracy and F1 score. section 8.2.2 section 9.2.3

optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

Shows the minimal PyTorch declaration connecting Adam to model parameters and a learning rate. section 8.2.2

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains Adam’s adaptive rates and momentum, then demonstrates configuration and training loops without deriving its update equations.
Examples
Demonstrates Adam with an AudioTransformer and an MLP trained on fake-voice and Elliptic Bitcoin datasets.
What it assumes
Assumes neural-network training, gradients, loss functions, and backpropagation while introducing optimizer selection through practical configurations.
What you can do after
Apply it via PyTorch by configuring Adam, running backpropagation, and monitoring validation and test metrics.
How it connects
Connects Adam to stochastic gradient descent, Adagrad, momentum, backpropagation, loss functions, and weight decay.

Also explained in

Compare how 10 books explain adam →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.