1 Data Engineering with Snowflake

 

This chapter covers

  • Featuring Snowflake for data engineering
  • Examining the responsibilities of a Snowflake data engineer
  • Constructing data pipelines with Snowflake
  • Data engineering with Snowflake applications

Organizations in just about every industry collect data, often in massive amounts. This data can be used by data analysts, data scientists, and other business professionals to extract insights that aid in decision making. Raw data is rarely suitable for consumption directly from the source. For most practical purposes, data must be transformed according to business rules, reshaped to fit a particular business need, and optionally enriched with external data before it is ready for downstream consumers.

Data engineering is the practice of building solutions that extract data from source systems, transform the data into useful information, and present the harmonized data to users for downstream consumption. The abbreviations ETL (extract – transform – load) or ELT (extract – load – transform) are also commonly used to denote these solutions.

Data engineers build pipelines that make data available to data analysts, data scientists, and other consumers who rely on this data to get their job done. Ensuring that high-quality data is delivered when users need it is crucial for successful analytics. Therefore, data engineers are an indispensable link in the data analytics realm.

1.1 Snowflake for Data Engineering

 
 

1.1.1 Snowflake Architecture

 
 
 
 

1.1.2 Snowflake Features for Data Engineering

 

1.2 Responsibilities of a Snowflake Data Engineer

 
 
 

1.2.1 Extracting Data from Source Systems

 
 
 

1.2.2 Performing Data Transformations

 
 

1.2.3 Presenting Data to Downstream Consumers

 
 
 

1.2.4 Applying Underlying Components

 
 
 

1.3 Building Data Pipelines

 
 
 

1.4 Data Engineering with Snowflake Applications

 
 
 

1.5 Summary

 
 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest