about this book
Data Pipelines with Apache Airflow, Second Edition, was written to help you implement data-oriented workflows (or pipelines) using Airflow. The book begins with the concepts and mechanics involved in programmatically building workflows for Airflow using the Python programming language. Then the book switches to more in-depth topics such as extending Airflow by building custom components and comprehensively testing your workflows. The final part of the book focuses on designing and managing Airflow deployments, touching on topics such as security and cloud platforms.
Who should read this book
Data Pipelines with Apache Airflow, Second Edition, was written for data professionals looking to develop basic workflows in Airflow and for those interested in advanced topics such as building custom components for Airflow and managing Airflow deployments. Because Airflow workflows and components are built primarily in Python, we do expect readers to have intermediate experience with programming in Python—that is, have a good working knowledge of building Python functions and classes, understanding concepts such as *args, **kwargs, and so on. Some experience with Docker is also beneficial because most of our code examples are run using Docker (though you can run them locally if you want to).