A failed model deployment is harder to reverse when the team cannot identify the previous checkpoint. A model registry records model versions and their training and evaluation metadata, providing an audit trail for deployment, reproducibility, and rollback. A lightweight registry can hold checkpoint paths rather than the model artifacts themselves. section 9.1.3 section 9.11
A JSON registry can track a version tag, training technique, base model, data hash, checkpoint path, evaluation metrics, and notes. Keep it in version control alongside the code so reviewers can inspect changes and scripts can parse the records. Register versions before deployment, even when there is only one model. section 9.1.2 section 9.1.3
A simple registry supports four operations:
- Register a uniquely tagged version with its checkpoint and metadata. section 9.1.2 section 9.1.3
- List versions and their statuses to inspect the deployment history. section 9.1.3
- Promote a registered version to active, retiring the version it replaces. section 9.1.2
- Roll back the most recent promotion, returning the prior version to active. section 9.1.2
Wire promotion and registry updates into the deployment script as a single step rather than relying on separate manual actions. Keep previous versions ready to serve: the registry identifies a rollback target, but recovery also depends on deployment infrastructure. section 9.1.3 section 9.11
Rollback after a safety regression
An IT support model fails a monthly bias and fairness check after a retrain. A registry rollback returns v1 to active and retires v2; recovery from alert to verified rollback takes under an hour. The team removes biased training examples, retrains v3, and validates it against the full safety suite before promotion. The retired v2 record remains available for audit. section 9.9.2
Table 9.2 Registry approaches by team scale
|
Approach
|
Best for
|
Tradeoffs
|
|
JSON file (this chapter)
|
Small teams, 1 to 10 models
|
Simple, version-controlled, no
infrastructure
|
|
MLflow / W&B Registry
|
Mid-size teams, 10 to 50 models
|
UI, search, comparison; requires a server
|
|
Vertex AI / SageMaker
|
Large teams, regulated industries
|
Full lifecycle management; cloud lock-in
|
|
DVC / lakeFS
|
Data-heavy pipelines
|
Data versioning; steep learning curve
|
|
Git LFS
|
Large model artifacts
|
Integrated with Git; storage costs scale
|
Compare registry approaches by team scale and operational tradeoffs. section 9.1.2
A single JSON file suits a small team starting out: it is easy to inspect, diff, and maintain without extra infrastructure. Consider a managed registry when version counts grow beyond roughly 10–15 or multi-team access control is needed. Managed options add capabilities such as dashboards and deployment integration, but may require a server or introduce cloud lock-in. section 9.1.2
