An InferenceService is a KServe resource that describes a deployed machine-learning inference service. Its specification identifies the model format and the location or image used to serve the trained model. section 9.3.1
A typical definition uses the serving.kserve.io/v1beta1 API, names the service, and configures a predictor with a TensorFlow model. A storage URI can point to a model directory on a persistent volume, while a serving image can provide the model server. section 9.3.1 section 9.4.1
The resource participates in an operational serving workflow:
- Create the service after training and model selection have produced the persisted model. section 9.4.1
- Check readiness with kubectl get isvc before sending requests. section 9.3.1
- Expose the ingress locally with port-forwarding, then call the /v1/models/<name>:predict endpoint using Python or curl. section 9.3.1
- Inspect the serving Pod logs to verify which model version TensorFlow Serving loaded. section 9.3.1
Flower-sample service
A service named flower-sample declares TensorFlow as its model format and reads saved_model_versions from the strategy-volume persistent volume. After creation, a prediction request returns class probabilities, while the logs confirm that TensorFlow Serving loaded version 4. section 9.3.1
Listing 9.21 Inference service definition
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
name: flower-sample
spec:
predictor:
model:
modelFormat:
name: tensorflow
storageUri: "pvc://strategy-volume/saved_model_versions"
Inspect the compact YAML resource definition for a TensorFlow-backed InferenceService. section 9.3.1
