Reproducibility means being able to recreate a model from an exact account of the components that produced it. Keeping the model binary is not enough: a rebuild may be needed on different computing infrastructure or with one problematic component replaced. section 4.3.3
Make reconstruction part of the project workflow:
- Identify and version the model and its other artifacts. Track the dependency graph that must accompany the model through testing and release. section 4.3.3
- Record experimental details and model-design decisions as work proceeds, so a useful result does not become an undocumented recipe. section 7.4.1
- Document the chosen tests and testing process well enough to reproduce them. The appropriate mix of tests depends on the project. section 8.1
The lost good model
A team develops a good model but loses part of its recipe during delivery. Its binary survives, yet the team cannot reliably recreate its behavior or replace one suspect component to check the result. Versioning the contributing artifacts prevents that gap. section 4.3.3
Reproducible records also reduce repeated work during exploratory modeling. They do not, by themselves, establish model quality: evaluation still needs care, including attention to data leakage. section 7.4.1
