Kubernetes is an open-source container orchestration platform for automating deployment, scaling, and management. It becomes useful when a service environment grows from a few containers to many containers distributed across servers, where scheduling, failover, restarts, monitoring, and completion tracking become operational burdens. section 1.2.5
A Kubernetes cluster turns server groups and their resources into manageable infrastructure. A deployment can request a Docker image with defined memory and GPU requirements, allowing Kubernetes to allocate a suitable runtime environment. section 1.2.5
For deep learning service environments, Kubernetes addresses several production requirements:
- Schedules containers across available servers instead of requiring services to select machines manually. section 1.2.5
- Supports operational management of server groups, including deployment, patching, and updates. section 1.2.5
- Provides namespaces as lightweight virtual clusters for separating users and training workloads on shared physical infrastructure. section 3.3.7
- Tracks namespace capacity and rejects training launches when configured CPU, memory, or GPU quotas are fully consumed. section 3.3.7
Submitting an isolated training job
An administrator creates a namespace with CPU, memory, and GPU quotas, then maps users to namespaces. When a user submits a training request, the training service identifies the assigned namespace and calls the Kubernetes API to place the executable there. Kubernetes tracks usage and rejects the launch if the namespace lacks capacity. section 3.3.7
Figure 3.10 Creating machine pools within the training cluster to set up the resource consumption boundary for different users.
Use the machine-pool boundary diagram to relate shared physical infrastructure to isolated resource consumption for different users. section 3.3.7
