A toleration is Pod scheduling configuration that permits a workload to run on a node with a matching taint. It does not require Kubernetes to place the Pod there. Taints keep ordinary workloads away from special nodes; tolerations identify workloads that can use them. section 8.1.3
Use tolerations when:
- A workload can run on lower-availability Spot nodes while other workloads should avoid them. section 8.1.3
- A workload requires access to nodes with special hardware such as GPUs. section 8.1.3
- Separate workload groups onto dedicated node pools, such as stable coordinator Pods and autoscaling batch workers. section 8.1.4
Tolerating a Spot taint
A node tainted with spot=true:NoSchedule rejects Pods without a matching toleration. A Deployment can add key spot and value "true" to its tolerations, allowing its replicas to be scheduled there. To ensure placement only on that node class, combine the toleration with a node selector. section 8.1.3
Separating workload groups
Taint one node with group=1:NoSchedule and label it group=1. A workload with a group=1 toleration and nodeSelector group: "1" can target that group while other workloads are kept away. A second group can use group=2. section 8.1.4
A Pod that remains Pending may be blocked by an unmatched taint. Inspect the Pod with kubectl describe Pod to read the FailedScheduling event; then add the required toleration or provide untainted nodes. section 8.3.2
Listing 8.5 Chapter08/8.1.3_Taints/deploy_tolerate_spot.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: timeserver
spec:
replicas: 3
selector:
matchLabels:
pod: timeserver-pod
template:
metadata:
labels:
pod: timeserver-pod
spec:
tolerations: #1
- key: spot #1
value: "true" #1
containers:
- name: timeserver-container
image: docker.io/wdenniss/timeserver:5
#1 This workload can tolerate nodes with the spot=true taint, and therefore may be scheduled on them.
Inspect a complete Deployment manifest that tolerates the spot=true taint. section 8.1.3
