Classification accuracy is the proportion of predictions that exactly match their ground-truth labels. It summarizes how often a classifier is correct across an evaluated dataset. section 3.4.3 section 13.4.3
Calculate accuracy through a short prediction-and-comparison pipeline:
- Run the evaluation dataset through the model and obtain class logits. section 3.4.3
- Select the class with the highest logit as each prediction. section 3.4.3
- Compare predictions with ground-truth labels, count exact matches, and aggregate the result across test batches. section 3.4.3 section 13.4.3
Two evaluation results
A trained ViT classifier reaches 0.5621 accuracy on its test evaluation. A fine-tuned ResNet50 reaches 0.9373 on a balanced test set containing 10,000 real and 10,000 fake images. section 3.4.3 section 13.4.3
Listing 3.10 Using the trained ViT to classify images
import math
import torch.nn.functional as F
model.load_state_dict(torch.load('files/ViT.pth'))
model.eval()
with torch.no_grad():
batch=next(iter(testloader)) #1
batch = [t.to(device) for t in batch]
images, labels = batch
logits, attention_maps = model(images,
output_attentions=True) #2
predictions = torch.argmax(logits, dim=1) #3
print(predictions)
print([names[i] for i in predictions.tolist()]) #4
Inspect a complete PyTorch evaluation path: load trained weights, switch to evaluation mode, obtain logits, select predictions with argmax, and display class names. section 3.4.3
