chapter seven
7 Triggering workflows with external input
This chapter covers
- Waiting for certain external conditions to be met with sensors
- Setting dependencies between tasks in different directed acyclic graphs (DAGs)
- Executing workflows via the REST API and the command-line interface (CLI)
- Triggering DAGs when messages are published on a message queue
In previous chapters, we saw how to trigger pipelines based on time intervals (chapter 3) and asset updates (chapter 4). In this chapter, we’ll explore ways to trigger workflows in response to events that occur outside Airflow, such as files being uploaded to a shared drive, developers pushing their code to a repository, and messages being published on a message bus. Any of these events could be reasons to start running your pipeline in a modern enterprise.