Suppose an Airflow task must fetch movie ratings from an API. Before a DockerOperator can run it, the task needs an image containing its script and dependencies. A Dockerfile is the text file of build instructions used to create that image. section 11.2.3 section 11.4.2
A small Python application can be packaged in a few steps:
- Start with a base image using FROM, such as python:3.12-slim. section 11.2.3
- COPY the application and, if needed, a requirements file into the image. Use RUN to install dependencies and make the script executable. section 11.2.3 section 11.4.2
- Set ENTRYPOINT to the executable and CMD to default arguments when the image should run the script directly. section 11.2.3
Listing 11.3 Dockerfile for fetching weather from the wttr.in API
FROM python:3.12-slim #1 COPY requirements.txt /tmp/requirements.txt #2 RUN pip install -r /tmp/requirements.txt COPY scripts/fetch_weather.py /usr/local/bin/fetch-weather #3 RUN chmod +x /usr/local/bin/fetch-weather ENTRYPOINT [ "/usr/local/bin/fetch-weather" ] #4 CMD [ "--help" ] #5
The weather-fetching Dockerfile shows the complete sequence: select Python, install requirements, copy the CLI script, make it executable, and set its default invocation. section 11.2.3
Run `docker build --tag manning-airflow/wttr-example .` from the directory containing the Dockerfile and its supporting files. The final dot supplies the build context, where Docker finds files named in COPY instructions. Test the result with `docker run manning-airflow/wttr-example:latest`; the example prints the script’s help message. section 11.2.3
For an Airflow workflow, build an image for each task that needs one, test it independently, then publish it to a container registry. DockerOperators reference those images; Airflow workers pull them and run containers with the task arguments. Keeping task software in images separates its development and testing from DAG development. section 11.4.2 section 11.4.4
A Dockerfile can also package DAG files into the Airflow deployment image with COPY. This makes the deployed DAG version identifiable, but changing a DAG then requires a new image build and deployment. section 17.2.7
