KServe
KServe is an open-source Kubernetes-native platform for deploying and serving machine-learning models through standardized resources and interfaces.
Designing Deep Learning Systems: A guide for software engineers
KServe is an open-source platform that provides standardized interfaces and infrastructure for deploying and serving machine-learning models on Kubernetes.
- How it is used
- Standardizes model deployment and inference across serving tools in Kubernetes platform scenarios.
- How deep it goes
- Explains interfaces, framework coverage, Knative responsibilities, and a deployment configuration while deferring detailed tool documentation.
- Examples
- Uses an InferenceService manifest that deploys a PyTorch predictor from a Google Cloud Storage model URI.
- What you can do after
- Operate/configure Kubernetes model serving through one KServe API across multiple frameworks and serving backends.
Read how Designing Deep Learning Systems: A guide for software engineers explains KServe →
Distributed Machine Learning Patterns
KServe is a Kubernetes-based model serving platform that provides standardized resources and abstractions for deploying machine learning models.
- How it is used
- Deploys and operates machine learning model servers in Kubernetes production workflows.
- How deep it goes
- Explains serving abstractions and operational features, but defers framework-specific server configuration and detailed deployment procedures.
- Examples
- Uses single-instance and replicated model servers, increasing request traffic, additional serving Pods, and a TensorFlow Fashion-MNIST workflow.
- What you can do after
- Operate/configure model deployments with autoscaling, replication, health checking, networking, and rollout support.
Read how Distributed Machine Learning Patterns explains KServe →
Related concepts
- Model Servingused by · A model-serving platform is software infrastructure that hosts trained models and exposes inference by accepting requests and returning model outputs.
- Kubernetesuses · Kubernetes is an open-source platform for orchestrating and automating the deployment, scaling, and management of containerized workloads across computing clusters.
- custom resource definitionproduces · A CustomResourceDefinition is a Kubernetes API object that defines a custom resource type and its schema.
- Knativeuses · Knative is an open-source, Kubernetes-based platform for deploying and operating serverless and event-driven workloads.
- BentoMLalternative to · BentoML is an open-source framework for packaging, deploying, and serving machine-learning models as production services or APIs.
- autoscalinguses · Autoscaling is the automatic adjustment of computing resource capacity or running instances in response to workload demand or measured system conditions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.