liveBook · Manning

Versioning

As explained in LLM Customization and Fine-Tuning

The practice of tracking distinct iterations of a model and their changes over time.

When a production model starts giving wrong answers, rollback depends on knowing what is running, what ran before it, and where the previous checkpoint lives. Versioning records distinct model iterations and their provenance so teams can reproduce decisions, revert deployments, and maintain an audit trail. section 9.1

A fine-tuned model version needs more than a prompt string: record the checkpoint, base model, training-data hash, and evaluation report. Preserve the prompt baseline, few-shot bank, held-out set, and validation harness alongside it so a rollback can restore the earlier approach. section 4.9.3

Table 9.1 Minimum fields in a model version registry entry
Field
Example
Why
Version tag
it-support-v2-20260215-lora-Qwen3-4B-v1
Human-readable identifier
Technique
lora, sft, distill, dpo
Tracks the training method
Base model
Qwen/Qwen3-4B-Instruct-2507
Required for reproducibility
Data hash
abc123
Ties the model to the exact training data
Checkpoint path
chapter06/runs/sft_run1
Where to find the weights
Eval metrics
{"overall_f1": 0.72, "safety_pass_rate": 1.0}
Performance at registration time
Status
registered, active, retired
Lifecycle state
Created timestamp
2026-02-15T19:36:04Z
Audit trail

Use these fields as a minimum registry entry; also retain the hyperparameters needed to reproduce the training run. section 9.1.1 section 9.1

A descriptive tag can encode the name, version number, date, technique, base model, and data version: {name}-v{N}-{date}-{technique}-{base_model_short}-{data_version}. The tag helps identify a candidate quickly; the registry entry holds the details and checkpoint location. section 9.1.1

Keep the registry lifecycle explicit:

  • Register each new version with its metadata and checkpoint path. section 9.1.2 section 9.1.1
  • List versions to see their status, then promote the selected version to active; the replaced version becomes retired. section 9.1.2
  • Roll back the most recent promotion when the new version degrades. section 9.1.2

Rolling back a degraded experiment

An IT-support registry records an active SFT baseline with F1 of 0.72 and safety pass rate of 1.0. A DPO experiment fell to 0.58 and 0.75 respectively and was retired after rollback. The entries retain data hashes and checkpoint paths, making the history inspectable. section 9.1.3

Start versioning before the first deployment, even with one model. A JSON file is easy to inspect, diff in a pull request, and commit alongside the code. As the team grows beyond roughly 10–15 model versions or needs multi-team access control, consider a managed registry; the fields and concepts remain useful while the storage backend changes. section 9.1.3 section 9.1.2

Table 9.2 Registry approaches by team scale
Approach
Best for
Tradeoffs
JSON file (this chapter)
Small teams, 1 to 10 models
Simple, version-controlled, no infrastructure
MLflow / W&B Registry
Mid-size teams, 10 to 50 models
UI, search, comparison; requires a server
Vertex AI / SageMaker
Large teams, regulated industries
Full lifecycle management; cloud lock-in
DVC / lakeFS
Data-heavy pipelines
Data versioning; steep learning curve
Git LFS
Large model artifacts
Integrated with Git; storage costs scale

Compare registry options by team scale and operational tradeoffs before choosing infrastructure. section 9.1.2

How LLM Customization and Fine-Tuning covers it

How deep it goes
Specifies registry fields, a naming scheme, lifecycle operations, and storage choices, with a JSON registry implementation.
Examples
Uses an IT-support registry to compare an SFT baseline with a degraded DPO experiment and its rollback.
What you can do after
Operate/configure a JSON registry by registering, listing, promoting, and rolling back model versions.
How it connects
Connects model registries with fine-tuning, prompt baselines, evaluation, drift detection, and rollback.

Also explained in

Compare how 36 books explain versioning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.