In an Airflow data pipeline, all_success keeps a downstream task from running until every upstream dependency finishes successfully. It is the default trigger rule, so ordinary task chains use it unless another rule is configured. section 6.4.1 section 6.3.2
Airflow evaluates the rule from upstream task states:
- A task with no unfinished unsuccessful dependencies becomes eligible to run. section 6.4.1
- If any dependency is skipped, the downstream task is skipped as well. section 6.3.2
- If any dependency fails, the downstream task cannot run and receives upstream_failed; the failure propagates downstream. section 6.4.2
- Independent branches can continue executing even when a task on another branch fails. section 6.4.2
Default behavior after a conditional task
latest_only >> deploy_model When latest_only raises AirflowSkipException, it receives skipped status and deploy_model is skipped under all_success. When latest_only succeeds, deploy_model becomes eligible to run. For a branch join where one path is intentionally skipped, use a different rule such as none_failed instead of leaving all_success in place. section 6.3.2 section 6.2.2
Figure 6.15 An upstream failure stops downstream tasks from being executed with the default trigger rule all_success, which requires all upstream tasks to be successful. Airflow continues executing tasks that don’t have any dependency on the failed task (fetch_weather and clean_weather).
Trace how an upstream failure stops dependent tasks while unrelated tasks continue. section 6.4.2
