Appendix A. Running code samples
This book comes with an accompanying code repository on GitHub (https://github.com/godatadriven/data-pipelines-with-airflow-2nd-ed). The repository holds the same code as demonstrated in this book, together with easily executable Docker environments so that you can run all examples yourself. This appendix explains how the code is organized and how to run the examples.
A.1 Code structure
The code is organized per chapter, and each chapter is structured the same. The top level of the repository consists of several chapter directories (numbered 01–17), which contain self-contained code examples for the corresponding chapters. Each chapter directory contains at least the following files/directories:
- dags—Directory containing the DAG files demonstrated in the chapter
- compose.yaml—File describing the standard Airflow setup needed for running the DAGs
- compose.override.yaml—File with chapter specific overrides and extra services
- .env—Chapter specific environment variables
- README.md—Readme introducing the chapter examples and explaining any chapter-specific details on how to run the examples
Where possible, code listings in the book will refer to the corresponding file in the chapter directory. For some chapters, code listings shown in the chapters will correspond to individual DAGs. In other cases (particularly for more complex examples), several code listings will be combined into one single DAG, resulting in a single DAG file.