liveBook · Manning

Restart policy

As explained in Kubernetes in Action, Second Edition

The OnFailure restart policy directs Kubernetes to restart containers when they terminate unsuccessfully.

A restart policy tells Kubernetes whether to replace a pod’s container after its process exits. Set `restartPolicy` in the pod spec: `Always` restarts after any exit and is the default; `OnFailure` restarts only after a non-zero exit code; `Never` does not restart. The pod-level setting applies to all its regular containers, not to each one separately. section 6.2.1

Let a Job retry a failed task

In a Job manifest, put `restartPolicy: OnFailure` under `spec.template.spec`, alongside the container definition. A Job pod must explicitly use `OnFailure` or `Never`: the default `Always` would prevent a successfully finished pod from completing. With `OnFailure`, the Kubelet replaces a failed container in the same pod; once the task succeeds, the pod can complete. section 18.1.1 section 18.1.3

Listing 18.1 A Job manifest for running a single task
apiVersion: batch/v1      #1
kind: Job          #1      
metadata:
  name: quiz-init
  labels:
    app: quiz
    task: init
spec:
  template:          #2
    metadata:                     #3
      labels:                     #3
        app: quiz                 #3
        task: init                #3
    spec:
      restartPolicy: OnFailure     #4
      initContainers:                                                      #5
      - name: init                                                         #5
        image: mongo:5                                                     #5
        command:                                                           #5
        - sh                                                               #5
        - -c                                                               #5
        - |                                                                #5
          mongosh mongodb://quiz-0.quiz-pods.kiada.svc.cluster.local \     #5
   --quiet --file /dev/stdin <<EOF                                   #5
                                                                           #5
          # MongoDB code that initializes the replica set                  #5
          # Refer to the job.quiz-init.yaml file to see the actual code    #5
                                                                           #5
          EOF                                                              #5
      containers:               #6
      - name: import            #6
        image: mongo:5          #6
        command:                #6
        - mongoimport                                                       #6
        - mongodb+srv://quiz-pods.kiada.svc.cluster.local/kiada?tls=false  #6
        - --collection                                                     #6
        - questions                                                        #6
        - --file                                                           #6
        - /questions.json                                                  #6
        - --drop                                                           #6
        volumeMounts:                                                      #6
        - name: quiz-data                                                  #6
          mountPath: /questions.json                                       #6
          subPath: questions.json                                          #6
          readOnly: true                                                   #6
      volumes:
      - name: quiz-data
        configMap:
          name: quiz-data

The Job manifest shows exactly where `restartPolicy: OnFailure` belongs in a pod template. section 18.1.1

“Restart” does not revive the old container: Kubernetes discards it and creates another. Data written only to its filesystem is lost. Repeated failures trigger increasing delays between attempts, which can reach five minutes; `CrashLoopBackOff` can appear while the container waits. If a regular container restarts, the pod’s init containers normally do not run again. section 6.2.1

The choice also determines who handles a failed Job task. With `OnFailure`, the Kubelet retries the container inside its existing pod. With `Never`, that pod fails and the Job controller creates another pod, possibly on a different node. A failing init container is likewise retried under `OnFailure` or `Always`; under `Never`, later containers never start. section 18.1.3 section 6.4.1

Do not confuse the pod’s policy with the `restartPolicy: Always` that can appear on an individual init container. In Kubernetes 1.28 and later, that setting marks a native sidecar, which stays running alongside the pod’s regular containers. section 5.5.4

How Kubernetes in Action, Second Edition covers it

How deep it goes
Defines all three policies and traces container recreation, retry delays, init-container failures, and Job-level failure handling.
Examples
Uses a Job manifest with `restartPolicy: OnFailure` and contrasts same-pod retries with replacement pods under `Never`.
What it assumes
Introduces the policy choices through pod processes and exit codes, then applies them to init containers and Jobs.
What you can do after
Operate/configure it by choosing a Job pod’s restart policy and identifying whether failed work retries in its pod or a replacement pod.
How it connects
Links restart policy to exit codes, init containers, Jobs, the Kubelet, the Job controller, and exponential back-off.

Also explained in

Compare how 3 books explain restart policy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kubernetes in Action, Second Edition. The section links point to the parts of the book it draws on.