liveBook · Manning

Kubernetes

As explained in Distributed Machine Learning Patterns

Kubernetes is an open-source platform for automating the deployment, scaling, and management of containerized applications.

Kubernetes, or K8s, is an open-source platform for automating deployment, scaling, and management of containerized applications. It uses declarative configuration to orchestrate containers across computing environments, reducing the direct complexity of container management. Its goal is to make complex distributed systems easier to deploy and manage while retaining the resource-utilization benefits of containers. section 8.2

For distributed machine learning, Kubernetes serves as the core infrastructure beneath higher-level tools. Kubeflow submits distributed training jobs to a Kubernetes cluster, while Argo Workflows constructs and submits the end-to-end workflow. This separates model and pipeline concerns from the environment that runs their containerized workloads. section 8.0 section 1.4

Figure 8.2 An architecture diagram of Kubernetes
08-02

The architecture diagram provides a component-level view of Kubernetes; operational work here focuses on using kubectl rather than studying those internal components. section 8.2

A local workflow establishes a cluster, isolates project resources, and inspects workloads:

  • Create a local cluster with k3d, which runs the lightweight k3s distribution in Docker and supports single-node or multinode development clusters. section 8.2.1
  • Inspect cluster nodes with kubectl to check readiness, roles, version, labels, and system information. section 8.2.1
  • Create a namespace to isolate a group of resources; resource names must be unique within that namespace, not across the cluster. section 8.2.1
  • Switch clusters and namespaces when working across contexts, optionally using kubectx and kubens to simplify navigation. section 8.2.1

Run and inspect a Pod

A Pod is the smallest deployable unit. A simple manifest defines a whalesay container that runs cowsay with “hello world.” Create it with kubectl, list Pods to inspect status, read its logs, and retrieve the generated YAML when deeper configuration details are needed. section 8.2.1

Kubernetes supports on-premises, hybrid, and public-cloud infrastructure, allowing workloads to move between environments. The practical emphasis is on managing local cluster resources and then using the cluster to run distributed machine-learning work, rather than on Kubernetes component internals. section 8.2 section 8.10

How Distributed Machine Learning Patterns covers it

How deep it goes
Defines orchestration and practices local cluster, namespace, and Pod operations, while deferring Kubernetes component internals.
Examples
Uses a k3d-created distml cluster, a basics namespace, and a whalesay Pod that prints “hello world.”
What it assumes
Assumes Python machine-learning experience, exposure to production applications, Bash scripting, and basic Docker knowledge.
What you can do after
Operate/configure a local Kubernetes cluster, isolate resources with namespaces, and create, inspect, and log a Pod with kubectl.
How it connects
Connects Kubernetes with Docker, Kubeflow, Argo Workflows, TensorFlow, containers, namespaces, Pods, and kubectl.
Cover of Distributed Machine Learning Patterns
liveBookmanning.com
Read Distributed Machine Learning Patterns on liveBook Get the book at manning.com

Also explained in

Compare how 22 books explain Kubernetes →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Distributed Machine Learning Patterns. The section links point to the parts of the book it draws on.