liveBook · Manning

Reproducibility

Reproducibility is the ability to obtain consistent or comparable results by repeating a process with the same documented inputs, methods, and relevant conditions.

How 11 Manning books explain it

Designing Deep Learning Systems: A guide for software engineers

The ability to repeat model training with the same inputs and obtain comparable results.

How it is used
Supports repeatable production training for debugging, troubleshooting, and trustworthy prediction-based decisions.
How deep it goes
Explains reproducibility through versioned datasets, tracked run inputs, workflow state, and production service responsibilities.
Examples
Uses versioned ImageNet experiments, static dataset snapshots, and an annotated Metaflow data-processing workflow.
What you can do after
Operate/configure reproducible training by versioning data and recording code, hyperparameters, workflow state, models, and artifacts.

Read how Designing Deep Learning Systems: A guide for software engineers explains reproducibility →

Developer Experience on Kubernetes

Reproducibility is the ability to obtain equivalent software or environment results from the same declared inputs.

How it is used
Supports consistent development environments and repeatable container-image builds for team workflows, security, and compliance.
How deep it goes
Explains input drift in Devcontainers and Dockerfiles, then demonstrates timestamp control as a Buildpacks reproducibility mechanism.
Examples
Uses apt-installed httpie, a devcontainer.json configuration, and an Appointments image built with Pack.
What you can do after
Evaluate/understand when declared environment and image inputs can produce changing outputs, and apply version pinning or Buildpacks accordingly.

Read how Developer Experience on Kubernetes explains reproducibility →

Infrastructure as Code, Patterns and Practices: With examples in Python and Terraform

Repeatability is the property of automation that produces consistent results when run multiple times.

How it is used
Uses configuration as the source of truth for creating, sharing, changing, and scaling infrastructure environments.
How deep it goes
Explains configuration drift, version control, immutability, parameterization, modules, and on-demand environments, without detailing a specific IaC implementation.
Examples
Uses development-environment recreation, network address changes, shared modules, versioned commits, and on-demand QA environments.
What you can do after
Evaluate IaC for reproducibility and apply version-controlled, parameterized configurations to recreate and remove environments.

Read how Infrastructure as Code, Patterns and Practices: With examples in Python and Terraform explains reproducibility →

Data-Oriented Programming: Reduce software complexity

Reproducibility is the ability to recreate the behavior of a computation in another environment using the same relevant inputs and conditions.

How it is used
Supports debugging and unit testing by replaying captured function contexts outside the original endpoint or environment.
How deep it goes
Explains deterministic arguments, serialization, REPL replay, test fixtures, and external-data limits, but does not develop formal theory.
Examples
Uses a nested book catalog, captured JSON arguments, REPL experiments, and unit tests for matching queries.
What you can do after
Diagnose/debug with it by capturing inputs, replaying calls, refining behavior, and converting scenarios into repeatable tests.

Read how Data-Oriented Programming: Reduce software complexity explains reproducibility →

Delivering Machine Learning Projects: From design to deployment

Reproducibility is the ability to recreate a computational result from the same specified inputs, components, and procedures.

How it is used
Supports rebuilding ML models and reproducing tests across experimental and delivery workflows.
How deep it goes
Defines reproducibility through artifact tracking and documents experimental and testing practices, without specifying a particular implementation.
Examples
Uses component replacement and a lab-book analogy to illustrate model recreation and experimental records.
What you can do after
Evaluate whether a model’s recorded components and testing procedures support recreation and investigation.

Read how Delivering Machine Learning Projects: From design to deployment explains reproducibility →

AI Model Evaluation

Reproducibility is the ability to repeat an evaluation and obtain comparable results from the same documented inputs, code, configuration, and procedures.

How it is used
Supports repeatable offline evaluations and traceable comparisons as models, data, and evaluation setups change.
How deep it goes
Provides artifact and documentation checklists and engineering rationale, without specifying a complete implementation.
Examples
Uses LLM judge run records, including dataset snapshots, sampling strategies, scoring rubrics, and human-alignment checks.
What you can do after
Operate/configure an evaluation workflow with versioned artifacts, documented runs, and traceable outputs.

Read how AI Model Evaluation explains reproducibility →

Machine Learning Platform Engineering

Reproducibility is the ability to obtain consistent results when repeating an operation under the same inputs, configuration, and environment.

How it is used
Supports reliable ML experimentation, deployment, auditing, and iteration by keeping data, code, configurations, and artifacts aligned.
How deep it goes
Explains versioning, automation, modular pipelines, and environment consistency, while deferring detailed algorithmic training mechanics.
Examples
Uses labeled dataset splits, automated experiments, Kubeflow preparation pipelines, YOLO training, and versioned RAG configurations.
What you can do after
Build it from scratch as a tracked, modular training workflow that executes consistently across environments.

Read how Machine Learning Platform Engineering explains reproducibility →

Machine Learning System Design

Reproducibility is the property of being able to recreate a dataset or computational result from its original source and documented procedure.

How it is used
Regenerates datasets reliably during data gathering, preprocessing, updates, and later project work.
How deep it goes
Defines documented and tested regeneration procedures, but stops short of prescribing a particular implementation.
Examples
Uses manually crafted datasets, omitted preprocessing steps, and regenerated batches as practical examples.
What you can do after
Build it from scratch through a documented, tested procedure using the original data source.

Read how Machine Learning System Design explains reproducibility →

Kubernetes for Developers

Reproducibility is the ability to recreate the same application environment from a recorded set of build instructions and inputs.

How it is used
Uses containers and declarative Kubernetes configuration to recreate development, staging, and production environments.
How deep it goes
Explains recorded build steps and contrasts declarative configuration with imperative commands, without detailing full GitOps practice.
Examples
Uses a pseudocode Dockerfile and timeserver deployment and exposure commands as concrete reproducibility examples.
What you can do after
Apply declarative Kubernetes configuration across environments and record container construction steps for repeatable deployment.

Read how Kubernetes for Developers explains reproducibility →

Architecting an Apache Iceberg Lakehouse

Reproducibility is the ability to obtain consistent analytical or computational results by repeating a process with the same inputs, versions, and conditions.

How it is used
Supports repeatable model training, policy reporting, historical analysis, and disaster recovery across lakehouse environments.
How deep it goes
Explains snapshot reads, metadata capture, replication, and path rewriting, but stops short of specifying a complete recovery runbook.
Examples
Uses notebook-based training workflows, MLflow outputs, cross-region backups, and preserved public-sector reporting views.
What you can do after
Apply snapshot-based reads and metadata controls to repeat training, reproduce historical reports, and reconstruct replicated tables.

Read how Architecting an Apache Iceberg Lakehouse explains reproducibility →

Build AI Drug Discovery Pipelines

Reproducibility is the ability to obtain consistent results by repeating an analysis with the same documented inputs, software, models, and procedures.

How it is used
Supports reliable repetition and validation of multimodal, end-to-end drug-discovery analyses.
How deep it goes
Specifies tracking models, datasets, preprocessing, and inference parameters, while emphasizing validation rather than detailing an implementation checklist.
Examples
Compares analyses using ESM-2 versions 1.0 and 2.0 and considers cascaded structure-prediction and docking accuracy.
What you can do after
Operate reproducible analyses by recording component versions, datasets, preprocessing, inference parameters, and end-to-end validation results.

Read how Build AI Drug Discovery Pipelines explains reproducibility →

Also covered in

What to know first

Related concepts

Questions about reproducibility

What is reproducibility?

Reproducibility is the ability to obtain consistent or comparable results by repeating a process with the same documented inputs, methods, and relevant conditions.

Which Manning books explain reproducibility?

Designing Deep Learning Systems: A guide for software engineers, Developer Experience on Kubernetes, Infrastructure as Code, Patterns and Practices: With examples in Python and Terraform, Data-Oriented Programming: Reduce software complexity, Delivering Machine Learning Projects: From design to deployment, AI Model Evaluation, Machine Learning Platform Engineering, Machine Learning System Design and 3 more.

What should I know before learning reproducibility?

Manning books that teach reproducibility usually introduce language model, automation, area under the ROC curve and compliance first.

What is reproducibility part of?

Reproducibility is part of infrastructure as code. Infrastructure as code is the practice of defining, provisioning, and managing computing infrastructure through machine-readable code or configuration rather than manual procedures.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.