Suppose an AR mask produces one ugly frame. Keeping the app running is probably better than shutting it down. Now suppose a self-driving car is unsure whether a pedestrian is nearby. Continuing with an uncertain answer is a very different choice. Robustness means trying to keep software operating despite errors, even if some results are inaccurate. Correctness favors returning no result over returning a wrong one. Choose between them by asking what a mistake would cost. section 2.4
For an ML model, robustness can also mean that a small input perturbation does not cause a large output change. A minor lighting change should not usually alter a computer-vision result; replacing a word with a synonym should not usually reverse a sentiment prediction. This is also called consistency, stability, or smoothness. The relevant question is not whether every input change is harmless, but which changes the product expects the model to tolerate. section 5.2.1
Check robustness beyond a single prediction:
- Compare outputs before and after small, relevant input changes. Measure variation when some change is acceptable; use a property-based test when the expected invariant should hold strictly. section 5.2.1 section 10.5.1
- Compare releases as well as inputs. Retraining should not make an unchanged user alternate unpredictably between fraudulent and legitimate classifications. section 5.2.1
- Inspect the feature pipeline. External sources, auxiliary models, complex dependencies, and low-quality data can make it fragile even when a feature improves prediction quality. section 11.5.1
Robustness is therefore a practical decision, not a blanket instruction to always return an answer. Set expectations from the cost of mistakes, then check whether inputs, retraining, and feature dependencies allow the system to meet them. section 2.4 section 5.2.1
