welcome

This book is intended for developers with a basic understanding of data warehousing, ETL, SQL, and, optionally, Python. It teaches data engineering principles through real-life examples that you can easily follow along, enabling you to build data pipelines in the Snowflake environment.

As a data engineer, your role is critical in extracting data from source systems, ingesting it into the Snowflake data platform, transforming it into meaningful information, and presenting the harmonized data to downstream consumers for analytics, reporting, and data science.

In this book, you’ll learn how to accomplish these tasks by leveraging Snowflake’s native functionality. Following the step-by-step examples, you will progress steadily, starting with a basic data pipeline with only a few components and gradually building on it with more advanced features.

In addition to the main components of data pipelines, data engineers must incorporate underlying components such as security, data governance, orchestration, continuous integration, and more. Drawing from my many years of experience building data warehousing solutions, I’ll share the best practices and recommendations for designing and building robust data pipelines that adhere to excellent software engineering principles while optimizing Snowflake consumption costs and improving performance.