Kubeflow
Kubeflow is an open-source platform for developing, deploying, and orchestrating machine-learning workflows on Kubernetes.
Machine Learning Platform Engineering
Kubeflow Pipelines is a Kubeflow component and software development kit for defining, executing, and orchestrating machine-learning pipelines.
- How it is used
- Uses Kubeflow as a Kubernetes ML platform and Kubeflow Pipelines to automate reusable ML workflows.
- How deep it goes
- Builds components, connects their artifacts and dependencies, compiles pipeline YAML, and runs workflows; also addresses notebook configuration and drift gating.
- Examples
- Uses a batch income-classifier pipeline and an object-detection data-preparation pipeline with dataset downloading and splitting.
- What you can do after
- Build it from scratch as a Kubeflow pipeline of reusable components, then compile, run, inspect, and extend it.
Read how Machine Learning Platform Engineering explains Kubeflow →
Distributed Machine Learning Patterns
An open-source platform for deploying and orchestrating portable, scalable machine learning workflows on Kubernetes.
- How it is used
- Submits and manages distributed machine-learning training jobs on Kubernetes as part of an automated workflow.
- How deep it goes
- Defines core terminology and provides hands-on job submission, while deferring complete implementation details to an end-to-end project.
- Examples
- Uses MNIST and Fashion-MNIST examples, Kubernetes clusters, TensorFlow training logic, and Argo Workflows pipelines.
- What you can do after
- Operate distributed TensorFlow training jobs on Kubernetes and connect them to machine-learning workflows.
Read how Distributed Machine Learning Patterns explains Kubeflow →
Related concepts
- ML Metadatauses · An open-source library for recording, storing, and querying metadata about machine learning workflows and their artifacts.
- Kubernetesuses · Kubernetes is an open-source platform for orchestrating and automating the deployment, scaling, and management of containerized workloads across computing clusters.
- Kubeflow training operatorhas part · The Kubeflow Training Operator is a Kubernetes operator that manages distributed and nondistributed machine learning training jobs through custom resources.
- machine learning infrastructurekind of · Machine learning infrastructure is the hardware, software, data, and services supporting the development, training, deployment, and operation of machine-learning systems.
- pipeline componentuses · A pipeline component is a discrete, executable stage in a data or machine-learning workflow that performs an operation and may consume or produce data artifacts.
- directed acyclic graphuses · A directed acyclic graph is a graph with directed edges and no directed cycles.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.