liveBook · Manning

Sparse_categorical_crossentropy

As explained in Deep Learning with Python, Third Edition

A classification loss function for integer-coded class targets that measures mismatch between predictions and true labels.

sparse_categorical_crossentropy is a classification loss for models whose targets are integer class labels. During training, it supplies the feedback signal used to adjust weight tensors and reduce prediction error through gradient descent. section 2.5 section 8.1

MNIST classification

A 10-output Dense layer with softmax predicts handwritten digits numbered 0 through 9. Because the MNIST labels are integers, compile the model with loss="sparse_categorical_crossentropy". Keras then uses that loss during mini-batch training with the Adam optimizer. section 8.1 section 2.5

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)

The compile call shows the loss in its practical setting: paired with Adam and accuracy before fitting the model. section 2.5

Use this rule of thumb:

  • Choose the sparse form when each target is represented by an integer class index rather than a one-hot vector. section 8.1
  • Pair it with a softmax output for multi-class classification, such as the ten-way MNIST classifier. section 8.1

How Deep Learning with Python, Third Edition covers it

How deep it goes
Explains its training role and label-format choice, but does not derive the loss mathematically or inspect its implementation.
Examples
Uses dense and convolutional MNIST classifiers with ten-way softmax outputs and integer digit labels.
What it assumes
Assumes familiarity with model compilation, softmax outputs, and integer labels while reinforcing how losses guide weight updates.
What you can do after
Apply it via Keras when compiling integer-label classifiers with softmax outputs.
How it connects
Connects sparse categorical crossentropy with softmax activation, Adam, mini-batch stochastic gradient descent, and integer labels.
Cover of Deep Learning with Python, Third Edition
liveBookmanning.com
Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.