liveBook · Manning

Versioning

As explained in AI Model Evaluation

Versioning is the practice of recording distinct revisions of data, models, code, prompts, metrics, and configuration so that changes can be identified and reproduced.

Versioning records distinct revisions of evaluation inputs and configuration so results can be traced and repeated. Without it, changes to a model, dataset, prompt, or metric can make comparisons inconsistent and undermine trust. section 2.2.2 section 9.10

Version and document the components that can affect an evaluation:

  • Datasets, model versions, code, metric definitions, and evaluation configuration. section 2.2.2
  • Prompts, rubrics, judge models, retrieval indexes, feature pipelines, and tool definitions. section 2.2.2 section 2.4

For an LLM judge, track prompt revisions and timestamps, and log response schemas and model version identifiers. Store prompt text and hashes in a versioned store such as DVC or Git LFS so a result can be traced to the prompt used. MLflow and similar tools can log prompt configurations and output metadata per run. section 9.10.1 section 9.10.4

Tracing a change in judge results

Even changing one adjective in a judge prompt can alter behavior. If scores shift, use prompt revisions and timestamps to identify which version was used. Re-run a fixed prompt set against a stored baseline to check for divergence in model or API behavior as well. section 9.10.1 section 9.10.4

Versioning supports reproducibility, but it is part of a broader evaluation system. Consistent metric computation, logging, orchestration, and monitoring are also needed to make runs reliable and useful at scale. section 2.4 section 9.10.4

How AI Model Evaluation covers it

How deep it goes
Provides a component checklist and prompt-tracking practices, without specifying a complete version-control implementation.
Examples
Uses prompt edits, timestamped revisions, response schemas, and fixed-prompt canary runs to illustrate traceability.
What you can do after
Diagnose/debug evaluation changes by tracing prompt versions and comparing fixed-prompt results with a stored baseline.
How it connects
Connects reproducibility with experiment tracking, prompt debugging, observability, and result drift.

Also explained in

Compare how 36 books explain versioning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.