Graph execution component

As explained in Designing Deep Learning Systems: A guide for software engineers

A serving-system component that executes a computation graph representing model operations and dependencies.

A graph execution component runs prediction workflows that require several model calls in sequence. It represents model operations and dependencies as a directed acyclic graph (DAG), then executes the chain as one request. section 6.3.3

A serving platform uses the component to:

• Describe dependencies between model predictions as a DAG. section 6.3.3
• Execute a multi-model prediction workflow in one go. section 6.3.3
• Work alongside routing components and inference-server backends that perform the individual predictions. section 6.3.3 section 6.4

Mortgage approval workflow

A mortgage application can flow through a PDF scanning model, a named entity recognition model, and a loan-scoring model. The graph execution engine runs these predictions in sequence as one approval request. section 6.3.3

Figure 6.7 A general prediction service (platform) design that works with arbitrary model types

Use the platform architecture to locate graph execution between the unified prediction interface, routing component, and model-serving backends. section 6.3.3

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Defines DAG-based sequential execution and stops short of detailing graph-engine internals.

Examples

Uses mortgage approval with PDF scanning, named entity recognition, and loan scoring models.

What it assumes

Assumes familiarity with model prediction requests and inference backends, while introducing DAG-based model chaining.

What you can do after

Evaluate/understand when a serving workflow needs DAG-based execution across multiple model predictions.

How it connects

Connects directed acyclic graphs, routing components, inference servers, model server backends, and unified prediction APIs.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

Related concepts

• directed acyclic graphuses · A directed acyclic graph is a graph with directed edges and no directed cycles.

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.