Calibration data matches the real-world inputs a model will encounter after deployment. Without that match, strong test-set accuracy can collapse when users provide different inputs. section 6.1.2
Professional food photos versus phone snapshots
A dish-recognition app trained on polished food photographs scored above 90% on its test set, yet failed on mobile photos of random restaurant dishes. The training images were not representative of production data. Collecting images directly from the deployment environment would address the mismatch. section 6.1.2
Use these rules when collecting calibration data:
- Collect data directly from the environment where the model will be used whenever possible. section 6.1.2
- Match the expected users and input sources: classify new IMDB reviews with IMDB reviews, and tweets with tweets from similar users. section 6.1.2





