Adam is an optimizer: it defines how a neural network changes its weight tensors after calculating loss gradients. It is commonly used as a reliable default, so choosing an optimizer need not become an early design obstacle. section 2.5 section 4.1.3
A practical compilation choice
A Keras model can be compiled with Adam alongside a task-appropriate loss. For binary classification, pair Adam with binary_crossentropy when the final layer produces a sigmoid probability. Training then applies the optimizer's update rules across mini-batches. section 4.1.3 section 2.5
model.compile( optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"], )
Inspect the compact Keras configuration that selects Adam, the loss function, and the accuracy metric together. section 2.5
Use Adam with these practical rules:
- Start with Adam when you need a broadly suitable optimizer for a new problem. section 4.1.3
- Choose the loss from the task and output representation; Adam does not replace that decision. section 4.1.3
- Remember that training still computes gradients through backpropagation and applies updates batch by batch. section 2.5
- Monitor performance on data outside the training set, because optimization can coincide with overfitting. section 4.1.7
