ML Metadata

As explained in Designing Deep Learning Systems: A guide for software engineers

An open-source library for recording, storing, and querying metadata about machine learning workflows and their artifacts.

ML Metadata is a lightweight library for recording and retrieving metadata from machine-learning workflows. Instrumenting pipeline steps preserves datasets, model files, algorithms, hyperparameters, metrics, versions, and relationships needed for experiment comparison, troubleshooting, lineage, and reproducibility. section 8.0 section 8.4.1

Each workflow step uses the MLMD client API against either a relational SQL backend or an MLMD gRPC server backed by SQL. Direct SQL access is simpler; the gRPC arrangement is recommended because it avoids exposing the database. section 8.4.1

Model the workflow with four core record types:

• Artifact: an input or output object, such as a dataset or model file. section 8.4.1
• Execution: a component or step in a workflow, such as a training run. section 8.4.1
• Event: a relationship connecting artifacts with executions. section 8.4.1
• Context: a logical group correlating artifacts and executions within one workflow. section 8.4.1

Register and query a training run

Create an Artifact with a dataset URI and properties such as day and split, create a RUNNING Execution, and register a model Artifact with its URI, version, and name. Query all artifacts, retrieve one by ID or URI, or filter properties with a condition such as a positive training day. section 8.4.1

Figure 8.6 Two different setups for instrumenting metadata with MLMD: (A) directly report metadata to the backend database and (B) report metadata to the gRPC server DB, database.

Compare direct client-to-database instrumentation with the recommended client-to-gRPC-server arrangement. section 8.4.1

MLMD fits systems that already provide artifact registries or metric visualization and need lightweight metadata integration. It can run independently, originated in TensorFlow Extended, and is used by Kubeflow for pipeline and notebook metadata. section 8.4.3 section 8.5

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Explains client instrumentation, SQL and gRPC deployment, record types, registration, retrieval, and filtering, but defers complete service implementation.

Examples

Uses dataset and model artifacts, a training execution, SQLite configuration, URI lookup, and property-based artifact filtering.

What it assumes

Assumes machine-learning workflows and artifacts, while defining metadata, executions, artifacts, events, and contexts.

What you can do after

Operate/configure MLMD by instrumenting pipeline steps, selecting a backend, registering records, and querying stored artifacts.

How it connects

Connects artifacts, executions, events, contexts, metadata stores, SQL, gRPC, TensorFlow Extended, Kubeflow, and MLflow.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

Related concepts

• Kubeflowused by · Kubeflow is an open-source platform for developing, deploying, and orchestrating machine-learning workflows on Kubernetes.
• librarykind of · A software library is a reusable collection of code or components that other programs can incorporate or invoke to provide functionality.
• MLflowalternative to · MLflow is an open-source platform for managing the machine-learning lifecycle, including experiment tracking, model and artifact management, and model deployment.
• gRPCuses · gRPC is an open-source remote procedure call framework for defining services and invoking them across process or network boundaries, commonly using Protocol Buffers and HTTP/2.

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.