liveBook · Manning

EmptyOperator

As explained in Data Pipelines with Apache Airflow, Second Edition

EmptyOperator is an Airflow operator that represents a task with no substantive work.

EmptyOperator represents a task with no substantive work. In a branched DAG, it makes the convergence point explicit by joining alternative paths before downstream processing continues. section 6.2.2

Join alternative ERP branches

After selecting either the old or new ERP path, add a join task: from airflow.providers.standard.operators.empty import EmptyOperator join_branch = EmptyOperator( task_id="join_erp_branch", trigger_rule="none_failed" ) [clean_sales_old, clean_sales_new] >> join_branch join_branch >> join_datasets The join uses none_failed so execution can continue when one branch is skipped, while keeping the trigger-rule adjustment localized to the join. section 6.2.2

Figure 6.10 To make the branching structure clearer, you can add an extra join task after the branch, tying the lineages of the branch together before continuing with the rest of the DAG. This extra task has an added advantage: you don’t have to change trigger rules for other tasks in the DAG because you can set the required trigger rule on the join task. That is, you no longer need to set the trigger rule for the join_datasets task.

Diagram the branch lineages converging through an explicit join task before the shared downstream task. section 6.2.2

How Data Pipelines with Apache Airflow, Second Edition covers it

How deep it goes
Defines the operator and demonstrates its wiring, but does not examine its implementation internals.
Examples
Uses old and new ERP branches converging before the join_datasets task.
What it assumes
Assumes familiarity with Airflow tasks, DAG dependencies, branching, and trigger rules.
What you can do after
Build a self-contained branch join that localizes none_failed behavior and preserves ordinary downstream dependencies.
How it connects
Connects branching, trigger rules, skipped tasks, and DAG dependencies.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data Pipelines with Apache Airflow, Second Edition. The section links point to the parts of the book it draws on.