A learning rate is a training setting that controls how far model parameters move during each update. In gradient descent, the optimizer scales each parameter’s gradient by this coefficient before adjusting the weights. section 4.1.1
Choose the value with training stability and convergence in mind:
- Set it too high and updates may overshoot the optimum or fail to converge. section 4.1.1
- Set it too low and convergence may become slow, or optimization may become stuck in a local minimum. section 4.1.1
- Treat it as an optimizer hyperparameter; Adam can use one learning rate while adaptive methods adjust update behavior across weights. section 9.2.3 section 4.1.1
Fraud-model training configuration
A PyTorch MLP for licit-versus-illicit Bitcoin transactions sets LR = 0.01 and passes it to Adam. The training loop backpropagates loss, calls optimizer.step(), and tracks validation metrics across 1,000 epochs. A separate audio-transformer example uses Adam with a learning rate of 0.001 for 10 epochs. section 9.2.3 section 8.2.2
WDECAY = 1e-4 optimizer = torch.optim.Adam(model.parameters(), lr=LR, weight_decay=WDECAY) #A
Inspect the optimizer configuration that passes the learning rate and weight decay to Adam. section 9.2.3






















