Some work has a clear finish line: clear a cache, process a batch of images, or run a maintenance command. A Kubernetes Job runs that work in Pods and tracks whether it completes. If a Pod fails or is evicted before finishing, the Job controller can schedule another attempt. Unlike a standalone Pod, a Job provides a controller responsible for completion. section 10.0 section 10.2
Choose a Job when the work should end. Choose a Deployment for workers that must keep running to handle an ongoing queue: a Deployment replaces a Pod that exits, while a Job treats a successful exit as completion. A CronJob can schedule Jobs for periodic work, such as daily cleanup. section 10.2 section 10.3.1 section 10.5
Run a maintenance command
Define a Job whose Pod runs a container command such as `python3 add_tasks.py`. The Pod gets its own resources instead of sharing those of a running application Pod through `kubectl exec`. Create it with `kubectl create -f job_addwork.yaml`, then use `kubectl get job,pods` to check for `1/1` completions and a `Completed` Pod. section 10.2.1
Listing 10.9 Chapter10/10.2.1_Job/job_addwork.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: addwork
spec:
backoffLimit: 2
template:
spec:
containers:
- name: pi-container
image: docker.io/wdenniss/pi_worker:2 #1
command: ["python3", "add_tasks.py"] #2
env:
- name: REDIS_HOST
value: redis-0.redis-service
- name: PYTHONUNBUFFERED
value: "1"
restartPolicy: Never
A one-off Job manifest shows the container command inside the Pod template, along with environment variables, a backoff limit, and `restartPolicy: Never`. section 10.2.1
For a finite batch, multiple Job Pods can work concurrently. Set `parallelism` to control how many workers run at once, and make each worker exit successfully when its work is done. A dynamic queue can use Redis for coordination; workers that exit when the queue is empty let the Job finish rather than keep consuming worker resources. A known, static set of tasks can instead use indexed Jobs without a separate queue data structure. A completed Job cannot simply scale back up; delete and re-create it to start a new run. section 10.3 section 10.3.1
Listing 10.12 Chapter10/10.3.1_JobWorker/job_worker.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: jobworker
spec:
backoffLimit: 2
parallelism: 2 #1
template:
metadata:
labels:
pod: pi
spec:
containers:
- name: pi-container
image: docker.io/wdenniss/pi_worker:3 #2
env:
- name: REDIS_HOST
value: redis-0.redis-service
- name: PYTHONUNBUFFERED
value: "1"
- name: COMPLETE_WHEN_EMPTY #3
value: "1"
restartPolicy: OnFailure
A two-worker Job manifest shows `parallelism: 2`, the setting that tells workers to finish when the queue is empty, and `restartPolicy: OnFailure`. section 10.3.1
