Model selection chooses the trained candidate that should advance to serving. In an end-to-end pipeline, several models train against the same task, compete on statistical performance, and a dedicated selection step promotes the top model for downstream predictions. section 7.1.2 section 7.3
Treat selection as a workflow dependency, not only an evaluation task. A sequential design waits for every training job before selection and serving can begin; one unusually slow job delays the whole pipeline. Asynchronous execution can let available models proceed earlier, but early candidates may be weaker than models that finish later. section 5.3.1 section 5.3.3
Choose the release policy according to user requirements:
- Serve an available trained model early when faster inference results matter more than waiting for the strongest candidate. section 5.7
- Wait for additional training jobs when a tolerable delay permits selection of better-performing models and stronger entity-tagging results. section 5.3.3
- Keep workflow steps reproducible and persist trained-model artifacts so the selected model is available to serving. section 7.1.2 section 9.4.1
Selecting a TensorFlow model for serving
Load each saved candidate from its versioned directory, evaluate it on testing data, compare accuracy, and copy the highest-accuracy model to a new versioned directory. The serving component then picks up that latest version. This selection script can run in a containerized Kubernetes Pod with the model volume mounted. section 9.2.3 section 9.2.3
Listing 9.16 Model evaluation
import numpy as np
import tensorflow as tf
from tensorflow import keras
import tensorflow_datasets as tfds
import shutil
import os
def scale(image, label):
image = tf.cast(image, tf.float32)
image /= 255
return image, label
best_model_path = ""
best_accuracy = 0
for i in range(1, 4):
model_path = "trained_model/saved_model_versions/" + str(i)
model = keras.models.load_model(model_path)
datasets, _ = tfds.load(
name='fashion_mnist', with_info=True, as_supervised=True)
ds = datasets['test'].map(scale).cache().shuffle(10000).batch(64)
_, accuracy = model.evaluate(ds)
if accuracy > best_accuracy:
best_accuracy = accuracy
best_model_path = model_path
destination = "trained_model/saved_model_versions/4"
if os.path.exists(destination):
shutil.rmtree(destination)
shutil.copytree(best_model_path, destination)
print("Best model with accuracy %f is copied to %s" % (
best_accuracy, destination))
The evaluation listing shows the full promotion loop: load candidate models, test each one, track the highest accuracy, and copy its artifacts to the serving version. section 9.2.3
A concrete pipeline trains a basic CNN, a CNN with dropout, and a CNN with batch normalization in parallel, then runs model selection before creating a KServe inference service. Pod cleanup after successful steps can release limited cluster resources, while retaining failed Pods supports debugging. section 9.2 section 9.4.1
