concept Docker container in category apache airflow

This is an excerpt from Manning's book Data Pipelines with Apache Airflow MEAP V05. Login to get full access to this book.

Now we have our basic rocket launch DAG, let’s get it up and running and view it in the Airflow UI. The bare minimum Airflow consists of two core components: (1) a scheduler and (2) a webserver. In order to get Airflow up and running, you can install Airflow either in your Python environment or run a Docker container. The Docker way is a one-liner:

Figure 8.4 Process of running a test with pytest-docker-tools. Running Docker containers during tests enables testing against real systems. The lifecycle of the Docker container is managed by pytest-docker-tools, and the user must implement the test.

[46] To ensure your tests run isolated from anything else, a Docker container with an empty initialized Airflow database can be convenient.