liveBook · Manning

Empirical audit

As explained in Sutskever's List

An empirical audit is a systematic evaluation of a claim or system against reproducible measurements and strong comparison baselines.

An empirical audit tests an elegant claim against measurable results, reproducible comparisons, and competitive baselines rather than reputation, hope, or paper appeal. section 7.6

Capsule networks

Capsules achieved strong results on smallNORB and near-best results on MNIST, but proved difficult to train and scale and failed to match leading CNNs on CIFAR and ImageNet. The comparison turned a promising architecture into a cautionary tale. section 7.6

How Sutskever's List covers it

How deep it goes
Contrasts benchmark outcomes with paper appeal but does not provide a detailed audit protocol or reproducibility checklist.
Examples
Compares capsule networks with CNNs across smallNORB, MNIST, CIFAR, and ImageNet.
What it assumes
Assumes familiarity with AI architectures, benchmarks, and state-of-the-art comparisons.
What you can do after
Evaluate whether a promising AI architecture survives comparison with competitive results.
How it connects
Connects benchmarks, baselines, neural network architectures, CNNs, and reasoning-oriented research.
Cover of Sutskever's List
liveBookmanning.com
Read Sutskever's List on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.