Evaluation mode

As explained in Designing Deep Learning Systems: A guide for software engineers

A model execution mode that disables training-specific behavior so a neural network can produce inference results.

Evaluation mode runs a neural network only to produce prediction results. Unlike learning mode, it does not update parameters such as weights and bias, making it the execution mode used for model serving. section 6.1.3

Use evaluation mode when converting a trained model into an operational prediction component:

• The predictor loads a model file from model storage, executes the network in evaluation mode, and returns the prediction. section 6.1.3
• A training container can become a predictor container by adding a web predict API and changing the neural network to evaluation mode. section 6.3.1
• The model can run inside a separate web service or directly within application logic on a user device. section 6.3.1

From training container to predictor

A single-model web application loads its model artifact when a request arrives, executes the neural network without updating parameters, and sends the prediction back through the web interface. section 6.3.1

Figure 6.3 A typical model prediction workflow in a prediction service

Follow the prediction workflow from request handling through model loading, evaluation-mode execution, and prediction response. section 6.1.3

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Contrasts parameter-updating learning mode with closed-loop execution and applies the distinction to predictor-container design.

Examples

Uses predictor containers, single-model web applications, and face-swap applications with locally embedded models.

What you can do after

Operate/configure a predictor by executing its trained neural network without parameter updates.

How it connects

Connects model serving with learning mode, prediction services, model training, web APIs, and direct model embedding.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA model setting used during inference or inspection that disables training-specific behavior such as dropout-like effects.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA model setting that switches layers such as dropout or batch normalization into inference behavior.

Compare how 3 books explain evaluation mode →

Related concepts

• dropoutconfigures · Dropout is a neural-network regularization technique that randomly suppresses units, connections, or activations during training to reduce overfitting.
• Model Servingused by · A model-serving platform is software infrastructure that hosts trained models and exposes inference by accepting requests and returning model outputs.
• Faster R-CNNused by · Faster R-CNN is an object-detection architecture that uses a learned region proposal network and detection heads to localize and classify objects in images.
• language modelconfigures · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• Gaussian processconfigures · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.
• epochcontrasts with · In machine learning, an epoch is one complete pass through the training data during model training.

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.