A persistent volume claim is a Kubernetes resource request for storage that containers can use beyond an individual task process. In an Airflow workflow running tasks as Kubernetes pods, the claim makes shared task data available through Kubernetes rather than through the Airflow worker. section 11.5.2 section 11.5.5
A claim specifies the storage class, access mode, and requested capacity. A matching PersistentVolume can provide the storage. For example, the data-volume configuration requests 1Gi with ReadWriteOnce and uses the manual storage class; the accompanying local volume maps storage to /tmp/data. section 11.5.2
Create storage for Airflow task pods
Apply the storage definition in the Airflow namespace: kubectl --namespace airflow apply -f resources/data-volume.yml. Kubernetes then creates data-volume as both a PersistentVolumeClaim and a PersistentVolume. Airflow task pods can mount the claim to store task results. section 11.5.2
A missing or incorrectly created claim prevents a task pod from starting. When logs show a pod stuck in Pending, inspect the pod with kubectl describe and read its Events. Applying the storage definition can resolve errors such as persistentvolumeclaim data-volume not found or an unbound claim. section 11.5.4
Listing 11.19 YAML specification for storage
apiVersion: v1
kind: PersistentVolume #1
metadata:
name: data-volume #2
Labels:
type: local
Spec:
storageClassName: manual
capacity:
storage: 1Gi #3
accessModes:
- ReadWriteOnce #4
hostPath:
path: "/tmp/data" #5
---
apiVersion: v1
kind: PersistentVolumeClaim #6
metadata:
name: data-volume #7
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce #8
resources:
requests:
storage: 1Gi #9
Inspect the complete PersistentVolume and PersistentVolumeClaim YAML, including storage class, capacity, access mode, and host path. section 11.5.2
