Metadata and artifacts store

As explained in Designing Deep Learning Systems: A guide for software engineers

A component that stores artifacts and descriptive metadata, including relationships among datasets, code, models, inferences, and metrics.

A metadata and artifacts store is the system record for outputs produced across a deep learning workflow. It stores artifacts—especially trained models—and their metadata, preserving lineage that supports experimentation and troubleshooting. section 1.4

Services write training inputs and results into this shared store: hyperparameters, code versions, training metrics, and test results. A user interface can then compare experimental runs, while product managers can query system metrics. section 1.2.3

In a production flow, the store connects code, training, and serving through a common handle:

• Register packaged training and inference code containers under the same handle so services can locate matching code. section 10.2.3
• Let the training service retrieve the training container, then save the trained model with its handle and metadata. section 10.2.4
• Let the model service query for the matching inference container and model file when an inference request arrives. section 10.2.5

Visual-recognition deployment

A visual_recognition handle identifies registered training and inference containers. Training uses the handle to find code and stores the resulting model; serving uses it to retrieve the model and inference container for a request. section 10.2.3 section 10.2.4 section 10.2.5

Figure 10.7 A typical production model training setup. The workflow service manages what, when, and how training flows are run. The training service runs model training jobs. The metadata and artifacts store provides training code, stores trained models, and associates them with metadata.

Follow the production training path: workflow orchestration selects a training flow, the trainer runs it, and the store supplies code and retains the model with metadata. section 10.2.4

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Defines component responsibilities and traces registration, training retrieval, model storage, and serving lookup without storage implementation details.

Examples

Uses a visual_recognition handle to register containers, train and store a model, then retrieve serving artifacts.

What it assumes

Assumes familiarity with training workflows, containers, and model serving while introducing the store's role in their production flow.

What you can do after

Operate/configure it by registering code containers, associating trained models with handles and metadata, and enabling serving-time artifact lookup.

How it connects

Connects workflow orchestration, model training, model serving, hyperparameter optimization, and model deployment through stored artifacts and metadata.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

Related concepts

• Model Servinguses · A model-serving platform is software infrastructure that hosts trained models and exposes inference by accepting requests and returning model outputs.
• model training serviceproduced by · A service that provides computation resources and job management for executing model-training code, producing trained models, and tracking training execution.

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.