When an Airflow deployment needs an operational view, Grafana turns collected metrics into dashboards for quickly understanding system behavior. A typical pipeline sends Airflow metrics through StatsD, translates them with a Prometheus StatsD exporter, stores them in Prometheus, and visualizes them in Grafana. section 15.7.1 section 15.7.4
A practical Grafana monitoring setup involves these steps:
- Run Grafana, for example with Docker, and open its local interface at port 3000. section 15.7.4
- Add Prometheus as a Grafana data source and provide its URL, such as http://localhost:9090 in a local setup. section 15.7.4
- Create a dashboard panel and query a Prometheus metric such as airflow_dag_processing_total_parse_time. section 15.7.4
- Interpret the resulting graph for changes or spikes that may indicate added DAG files, scheduler problems, or a problematic DAG file. section 15.7.4
Visualizing Airflow DAG-processing time
Start Grafana with `docker run -d -p 3000:3000 grafana/grafana`. Configure Prometheus as the data source at `http://localhost:9090`, create a dashboard panel, and query `airflow_dag_processing_total_parse_time`. The graph shows the seconds required to process all DAGs; a large spike can signal scheduler or DAG-file trouble. section 15.7.4
