6 Triggering workflows with external input
This chapter covers
- Waiting for certain external conditions to be met with sensors
- Deciding how to set dependencies between tasks in different DAGs
- Executing workflows via the CLI and REST API
- Triggering DAGs when messages are published on a Message Queue
In chapter 3, we explored how to schedule workflows in Airflow based on a time interval. The schedule can be defined with fixed triggers (e.g. CronTriggerTimetable) or intervals (e.g. CronDataIntervalTimetable). The different types of triggers are all intended to set the workflow to trigger at a certain time or interval. Airflow will compute the next time to run the workflow given the interval and start the first task(s) in the workflow at the next date and time.
In chapter 4, we explored other mechanisms for triggering workflows. The focus here was on event-driven scheduling, where no fixed interval or time was defined but a specific event occurred which caused an Airflow workflow to run.