A smoke test is a fast preliminary check that a system operates well enough to justify deeper testing. For an ML training pipeline, run it after changes on a small dataset, for few epochs, and possibly with a reduced model. The test should complete without errors and produce reasonable output, such as decreasing loss. section 10.5
Use smoke tests to:
- Catch regression bugs early and increase iteration speed during experimentation and debugging. section 10.5
- Check both that the pipeline runs and that its output is plausible, rather than proving every component correct. section 10.5
- Pair a high-level pipeline smoke test with low-level unit tests for the most sensitive components. section 10.5
- Treat maintenance as a tradeoff: significant pipeline changes can require corresponding smoke-test updates. section 10.5
Training-pipeline check
A test replaces a full ResNet with a dummy model, points training at a fixture dataset, limits training to one epoch, and asserts that training and validation loss remain below thresholds. It also verifies that the expected model constructor is called once.
The same idea appears at system level in a baseline: use the easiest workable alternative to check that components connect and produce the required format before investing in a sophisticated model. Unlike a disposable prototype, the baseline can remain as a comparison point and fallback response. section 8.1
