Versioning records distinct revisions of evaluation inputs and configuration so results can be traced and repeated. Without it, changes to a model, dataset, prompt, or metric can make comparisons inconsistent and undermine trust. section 2.2.2 section 9.10
Version and document the components that can affect an evaluation:
- Datasets, model versions, code, metric definitions, and evaluation configuration. section 2.2.2
- Prompts, rubrics, judge models, retrieval indexes, feature pipelines, and tool definitions. section 2.2.2 section 2.4
For an LLM judge, track prompt revisions and timestamps, and log response schemas and model version identifiers. Store prompt text and hashes in a versioned store such as DVC or Git LFS so a result can be traced to the prompt used. MLflow and similar tools can log prompt configurations and output metadata per run. section 9.10.1 section 9.10.4
Tracing a change in judge results
Even changing one adjective in a judge prompt can alter behavior. If scores shift, use prompt revisions and timestamps to identify which version was used. Re-run a fixed prompt set against a stored baseline to check for divergence in model or API behavior as well. section 9.10.1 section 9.10.4
Versioning supports reproducibility, but it is part of a broader evaluation system. Consistent metric computation, logging, orchestration, and monitoring are also needed to make runs reliable and useful at scale. section 2.4 section 9.10.4



































