Model type

As explained in Designing Deep Learning Systems: A guide for software engineers

A category of machine learning models sharing a particular input/output format and data-processing logic.

A model type categorizes models by their input/output format and data-processing logic. Different types usually require distinct interfaces, preprocessing, postprocessing, and caching behavior. section 6.3.2

Design implications include:

• Run separate prediction services when model types have substantially different interfaces or processing logic. section 6.3.2
• Give each service its own model cache and model-swapping behavior. section 6.3.2
• Reconsider this architecture when supporting many model types because service deployment, monitoring, and maintenance costs accumulate. section 6.3.2

Image and intent classification

An image classification service and an intent classification service can remain separate because their input/output formats and processing logic differ. section 6.3.2

Figure 6.6 A prediction service with model caching for multitenant applications

Use the prediction-service architecture to inspect how model caching supports multitenant requests. section 6.3.2

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Explains architectural consequences and stops short of detailing implementation for every model type.

Examples

Uses image classification and intent classification as contrasting model types requiring separate prediction services.

What you can do after

Evaluate whether model types should share a service or use separately managed prediction services.

How it connects

Connects model types to model caching, prediction services, web interfaces, preprocessing, postprocessing, and multitenant applications.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.