Poke mode keeps a sensor in its task slot while it repeatedly checks whether an external condition is true. This is useful for waiting on data, but prolonged polling can consume available task capacity. section 7.1.2
Waiting for supermarket data
A daily workflow checks whether each supermarket has delivered its data. If several deliveries are missing, poke-mode sensors continue polling and occupy task slots across successive DAG runs. The resulting buildup can block unrelated tasks. section 7.1.2
Manage the operational cost of polling by:
- Setting a timeout so a sensor eventually fails instead of waiting indefinitely. section 7.1.2
- Limiting concurrent DAG tasks with max_active_tasks when sensor activity could exhaust capacity. section 7.1.2
- Considering mode="reschedule" when sensors should release their slot between checks. section 7.1.2
Figure 7.8 Sensor deadlock: all the running tasks are sensors waiting for a condition to be True, which never happens; thus, they occupy all slots.
Shows how sensors waiting for a never-true condition can occupy all available task slots. section 7.1.2
