kubectl is the command-line client for querying and managing Kubernetes clusters. In an Airflow deployment, it helps prepare namespaces, storage, services, and task pods before workflows run. section 11.5.2 section 11.5.1
A practical kubectl workflow includes:
- Verify cluster access with kubectl cluster-info before creating resources. section 11.5.2 section 17.2.2
- Create an Airflow namespace with kubectl create namespace airflow. section 11.5.2
- Apply YAML specifications for persistent storage, claims, deployments, and services with kubectl apply -f. section 11.5.2
- Inspect task pods with kubectl get pods, then investigate scheduling failures with kubectl describe pod. section 11.5.4
Diagnosing a pending Airflow task pod
When a task remains pending, first identify its pod with kubectl --namespace airflow get pods. Describe that pod to inspect recent events. A FailedScheduling event reporting a missing persistentvolumeclaim points to a missing storage resource; apply resources/data-volume.yml, then describe the pod again to confirm scheduling. section 11.5.4
Listing 11.20 Deploying the storage resources using kubectl
$ kubectl --namespace airflow apply -f resources/data-volume.yml persistentvolumeclaim/data-volume created persistentvolume/data-volume created
The command listing shows how kubectl applies a storage specification inside the Airflow namespace and reports the created resources. section 11.5.2
