A worker performs the work in an Airflow pipeline. The DAG processor reads the DAG definition, and the scheduler determines when tasks are ready and queues them after their upstream dependencies complete. Workers then pick up tasks and execute them; the scheduler does not do the task’s work itself. section 1.2.3
Workers belong to the execution framework defined by the executor. With CeleryExecutor, for example, they are Celery worker processes. EdgeExecutor instead supports an Edge Worker in an environment outside the Airflow instance. This is useful when a task must run on premises while Airflow runs in the cloud, or within a particular Kubernetes cluster. section 15.1 section 15.2.3
Figure 15.6 With the EdgeExecutor, the worker runs in a separate environment, outside the environment in which Airflow runs.
The worker is shown outside Airflow’s environment—the placement that distinguishes EdgeExecutor from the other execution arrangements. section 15.2.3
An Edge Worker registers with the API server when it starts. The API server assigns it tasks, and the worker notifies the API server when they finish. Before using one, install the edge3 provider across the Airflow environment, configure the full EdgeExecutor path, set the shared internal API secret, enable the Edge Worker API endpoint, and set the worker’s API URL. The shared secret and executor configuration are operational considerations: this executor is described as experimental. section 15.2.3
Route one task to an Edge Worker
A DAG can give a BashOperator task an explicit `executor='airflow.providers.edge3.executors.EdgeExecutor'` while leaving a PythonOperator task without that override. After starting an Edge Worker, use `airflow edge list-workers` to inspect worker status. section 15.2.3
