Suppose Airflow runs in the cloud, but a DAG task must run on premises. EdgeExecutor lets a worker run outside the environment hosting the other Airflow components, so that task can run where it needs to. section 15.2.2 section 15.2.3
The remote executor needs the Airflow API server URL and an API token to communicate with it. EdgeExecutor is distributed, but its fit is location-specific: CeleryExecutor and KubernetesExecutor are alternatives when the main goal is scaling tasks across machines. section 15.2.2
Select EdgeExecutor for a task
A task can name EdgeExecutor explicitly: hello = BashOperator( task_id="hello", bash_command="echo 'hello'", executor='airflow.providers.edge3.executors.EdgeExecutor' ) This leaves room for another task in the DAG to use a different executor. section 15.2.3 section 15.10
For an installation, check these points:
- Install the Edge provider with `pip install apache-airflow-providers-edge3`; Airflow also exposes `AIRFLOW__CORE__EXECUTOR` for selecting an executor. section 15.2.3
- Run a DAG to validate executor pickup: a task reaching `running` has passed through scheduling and queuing. section 15.2.3
- Allow for experimental status, logging limitations, and untested performance when assessing production use. section 15.10 section 15.2.2
Figure 15.6 With the EdgeExecutor, the worker runs in a separate environment, outside the environment in which Airflow runs.
The diagram locates the EdgeExecutor worker outside Airflow’s main environment—the distinction that makes it useful for location-specific tasks. section 15.2.3
