Underconfiguration makes an ML training pipeline too rigid to change. Early prototypes often hardcode model architecture, datasets, preprocessing, or training settings because a small pipeline seems easy to understand without software abstractions. section 10.4
Recognize underconfiguration when:
- Changing a dataset, preprocessing step, or model architecture requires finding scattered hardcoded values. section 10.4
- The pipeline’s structure is convoluted enough that even simple alterations are difficult to understand or implement. section 10.4
- The remedy is not to expose everything: make changes with high likelihood configurable, while avoiding unnecessary abstraction layers. section 10.4
- Test configuration changes to catch regression bugs, increase iteration speed, and improve the pipeline’s design. section 10.7
Rigid training settings
A training function that embeds batch_size = 32 and learning_rate = 3e-4 directly in its body illustrates rigid configuration. Altering these choices requires editing implementation code rather than supplying pipeline settings.
