The data interval end is an Airflow context value identifying when a DAG run’s data interval ends. Interval-based schedules execute after that endpoint has passed, allowing tasks to process a known time range through data_interval_start and data_interval_end. section 3.4.4 section 3.8
Use data_interval_end to:
- Identify the interval whose data an incremental-processing task should handle. section 3.4.4
- Distinguish an interval boundary from logical_date, which generally points to the interval start and can identify the run. section 3.4.4
- Coordinate deployment logic with the current and next interval boundaries when backfills could otherwise deploy many historical models. section 6.3.1
Restrict deployment to the latest run
A deployment task can compare its execution time with the next interval returned by the DAG timetable. If the current interval is the final scheduled interval, or execution occurs between the next interval’s start and end, the task deploys; otherwise, it skips deployment. This workaround uses next_dagrun_info because next_data_interval_end is not available directly in the Airflow context. section 6.3.1
Figure 3.8 Time represented in terms of Airflow’s scheduling intervals, assuming a daily interval with a start date of 2024-01-01. Execution happens at the end of the interval (e.g., the DAG run for the first interval is executed at 2024-01-02 00:00).
A daily timeline shows how an interval beginning on 2024-01-01 is executed after its endpoint, around 2024-01-02 00:00. section 3.4.4
