Accuracy is the fraction of evaluated predictions that match their target labels. If a digit classifier gets 9784 of 10,000 test images right, its accuracy is 0.9784, or 97.84%. It measures correct classifications, not how confident the predictions were. section 2.1
Measure a digit classifier
In R/Keras, compile a classifier with `metrics = "accuracy"`, then train it with `fit()`. Training displays both loss and accuracy. After training, `metrics <- evaluate(model, test_images, test_labels)` and `metrics$accuracy` report accuracy on images held out for testing; the MNIST example returns `0.9784`. Loss guides training, while accuracy lets you monitor correct classifications. section 2.1
Read accuracy alongside the data split. Training accuracy describes predictions on training examples; validation accuracy helps assess performance during development; test accuracy checks the finished model on held-out examples. If validation results worsen while training continues, more epochs need not help: a 46-class news classifier begins to overfit after nine epochs, so it is retrained for nine before test evaluation. section 2.1 section 4.2.4
An accuracy number needs a point of comparison:
- Compare it with a trivial baseline. Random guessing gives about 50% on balanced binary classification and 10% on ten equally likely digit classes. section 4.2.4 section 5.2.2
- Check class balance before interpreting it. For the 46-class news task, an empirical random baseline is about 19%, not 1 divided by 46. If 90% of binary examples share one label, always predicting that label already reaches 90% accuracy. section 4.2.4 section 5.2.2
- Choose the metric to fit the goal. Accuracy is appropriate for a balanced cats-versus-dogs dataset; precision, recall, or counts of prediction outcomes may serve class-imbalanced or multilabel problems better. section 8.2.2 section 6.1.4
Listing 2.7 Evaluating the model on new data
metrics <- evaluate(model, test_images, test_labels)
metrics$accuracy
[1] 0.9784
The test-evaluation listing shows the exact R/Keras call and the resulting accuracy value. section 2.1
