Human-level performance is a benchmark threshold based on people completing a specified task. For ImageNet image labeling, a 2014 experiment with two expert annotators reported a 5.1% error rate; 5% was then used as the human threshold. section 3.6
That threshold is a shaky measure of general human ability: it rests on two annotators, trained on 500 and 100 images. It also compares humans with a machine-oriented top-5 metric. ResNet’s top-1 accuracy was 78.57%, below the 95% threshold. section 3.6
A top-5 prediction
For a basketball image, a model might guess “croquet ball,” “bikini,” “warthog,” “basketball,” and “moving van.” Top-5 scoring counts this as correct because the true label appears among the guesses—a rule that fits human judgments awkwardly. section 3.6
