1 Data Engineering with Snowflake
This chapter covers
- Featuring Snowflake for data engineering
- Examining the responsibilities of a Snowflake data engineer
- Constructing data pipelines with Snowflake
- Data engineering with Snowflake applications
Organizations in just about every industry collect data, often in massive amounts. Data analysts, data scientists, and other business professionals use this data to gain insights that aid decision-making. Raw data is rarely suitable for consumption directly from the source. Consumers require data that is transformed according to business rules, reshaped to fit a particular business need, and optionally enriched with external data.
Data engineering is the practice of building solutions that extract data from source systems, transform the data into useful information, and present the harmonized data to users for downstream consumption. The abbreviations ETL (extract – transform – load) or ELT (extract – load – transform) are also commonly used to denote these solutions.
Data engineers are responsible for building data pipelines that enable data analysts, data scientists, and other users to access the data they need to do their jobs. Providing high-quality data on time is essential for effective analytics, so data engineers play a critical role in the data analytics domain.