A model update can change predictions even when the calling code stays the same. Assigning a distinct version to each model revision makes that change visible and lets callers request a specific revision. In an API, loading the requested version when a model is initialized is one straightforward approach. section 13.1
Do not replace a model while leaving its version unchanged. That can change system behavior without notifying callers, and some model updates are breaking changes. Check the rest of the prediction pipeline too: changes to input and output data, preprocessing, postprocessing, or dependencies can also affect compatibility, sometimes unintentionally. section 13.1
Listing 13.3 Adding a model version to the API
class Model:
def __init__(self, version):
self.version = version
self.model = load_model(version)
def predict(self, features, time=None, seed=42):
if time is None:
time = datetime.now()
return self.model.predict(features, time, seed)
The Model constructor accepts a version and passes it to load_model; predictions then use the loaded revision. section 13.1
A dependency change that changed outputs
A Python-version upgrade produced mismatched outputs in a system that required full compatibility, despite everything else being held static. The lesson is to test behavior, not just whether the model identifier stayed the same. section 13.1
Model versions are only part of a reproducible result. If a feature calculation or its data source changes, old and new values should not be mixed without distinction. Dataset metadata can record feature versions alongside the features and timestamp range, making it possible to return to an older dataset and reproduce an earlier model result. section 11.4.2
