A weather dashboard needs to fetch data, clean it, and push the result. Draw each task as a node and each dependency as an arrow: fetch → clean → push. An arrow means the task at its tail must finish before the task at its head can start. This directed graph is acyclic because following its arrows cannot lead back to an earlier task. section 1.1.1
The absence of cycles makes the pipeline executable. If two tasks each require the other to finish first, neither can start. To run a DAG, find unfinished tasks whose upstream dependencies are complete, queue them, execute them, and repeat. In the dashboard pipeline, fetch becomes ready first, then clean, then push. Independent tasks can run in parallel. section 1.1.1 section 1.1.2 section 2.2
Figure 1.4 Using the DAG structure to execute tasks in the data pipeline in the correct order. The figure depicts each task’s state during each loop through the algorithm, demonstrating how it leads to the completed execution of the pipeline (end state).
Trace which dashboard task is ready on each pass through the dependency-checking procedure. section 1.1.2
In Apache Airflow, a DAG is also the Python-defined workflow that groups tasks, their dependencies, and scheduling metadata. Airflow parses the DAG file; its scheduler checks when a run is due and which tasks are ready, while workers perform the tasks. A DAG can run on a recurring schedule or be triggered without one. Python is Airflow’s primary DAG language, although Airflow 3 introduced an option to use other languages. section 1.2.1 section 1.2.3 section 2.2
Define a small task chain
For a rocket-launch workflow, create tasks to download launch data, retrieve pictures, and notify. Express their order as `download_launches >> get_pictures >> notify`. Airflow supports an explicit DAG object, a context manager, or a Taskflow API `@dag` decorator. Choose task boundaries deliberately: a single large task hides steps, while splitting every action may add unnecessary tasks. section 2.2 section 6.6.2
DAGs suit recurring, batch-oriented workflows better than per-event streaming. Code-defined workflows offer flexibility, but large DAGs need modular tasks and maintainable structure. To check a whole workflow, `dag.test()` can run a DAG programmatically; account for external systems in the test. The shown `dag.test()` approach has a reported DAG-serialization issue in Airflow 3.1. section 1.3.2 section 10.4.1
