How deeply does Deep Learning with Python, Third Edition cover TensorFlow Serving?

Defines graph-and-weights serving and demonstrates export, reload, and prediction, but defers server configuration and operational internals.

TensorFlow Serving

As explained in Deep Learning with Python, Third Edition

A serving library for deploying TensorFlow computation graphs with saved weights as inference services.

TensorFlow Serving deploys a TensorFlow computation graph together with saved weights as an inference service. It moves the model outside the Python program, allowing applications written in JavaScript, C++, or other environments to consume predictions through a REST API. section 6.3.2

Exporting a Keras model

Export a Keras model in TensorFlow SavedModel format, reload the artifact, and call its serving endpoint with input data. This separates production inference from training and creates room for speed and memory optimizations. section 6.3.2

model.export("path/to/location", format="tf_saved_model")             #1

reloaded_artifact = tf.saved_model.load("path/to/location")            #2
predictions = reloaded_artifact.serve(input_data)                       #2

The listing shows the complete export, reload, and prediction flow for a TensorFlow Serving artifact. section 6.3.2

Use this deployment pattern when:

The consuming application can reliably reach a remote service over the internet. section 6.3.2
Roughly 500 milliseconds of request, inference, and response time is acceptable. section 6.3.2
Inference inputs are not highly sensitive, because the server must access them in decrypted form; use SSL for HTTP traffic. section 6.3.2

How Deep Learning with Python, Third Edition covers it

Cover of Deep Learning with Python, Third Edition

liveBook manning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Deep Learning with R, Third EditionTensorFlow Serving is a system for serving trained machine-learning models through production prediction APIs.

Compare how 2 books explain TensorFlow Serving →

Related concepts

NVIDIA Triton Inference Servicealternative to · NVIDIA Triton Inference Server is open-source software for deploying and serving machine-learning models through standardized interfaces across frameworks and hardware backends.
ONNXalternative to · ONNX is an open standard for representing machine-learning models in a portable format that supports exchange and execution across frameworks and runtimes.
KServeused by · KServe is an open-source Kubernetes-native platform for deploying and serving machine-learning models through standardized resources and interfaces.
Model Servingused by · A model-serving platform is software infrastructure that hosts trained models and exposes inference by accepting requests and returning model outputs.
TensorFlowuses · TensorFlow is an open-source software platform for numerical computation and machine learning, with tools for developing, training, and deploying models.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.