ETL

As explained in Snowflake Data Engineering

ETL is a data integration technique that extracts data, transforms it, and then loads it into a target system.

ETL is a data pipeline pattern that extracts data from operational systems or files, formats, cleans, and transforms it according to business requirements, then loads the results into target tables in Snowflake. section 11.1.2

ETL is particularly useful when:

• The target data warehouse uses a different model from the source system and requires substantial transformation. section 11.1.2
• Complex transformations should be completed before loading, reducing transformation work for downstream consumers. section 11.1.2
• Teams can accept the operational costs of transformation during extraction: processing may be time-consuming, resource-intensive, and difficult to maintain when embedded in extraction tools or custom applications. section 11.1.2

Bakery pipeline

A bakery pipeline can extract partner and baked-goods data from operational tables, clean and reshape it according to business requirements, and load the transformed results into Snowflake target tables. Outputs can be organized across extraction, staging, data warehouse, and presentation layers. section 11.0 section 11.3

Figure 11.1 ETL pattern where data is first extracted from the source systems, then transformed according to business requirements, and finally loaded into Snowflake

Visualizes the ETL sequence: extract from source systems, transform according to business requirements, and load into Snowflake. section 11.1.2

How Snowflake Data Engineering covers it

How deep it goes

Defines the three-stage sequence and weighs suitability against processing cost, maintenance burden, and loss of pre-transformation data.

Examples

Uses bakery partner and baked-goods tables sourced from operational systems, with transformed outputs organized into Snowflake pipeline layers.

What it assumes

Assumes familiarity with source systems, target tables, and data pipelines while grounding the pattern in operational databases and files.

What you can do after

Evaluate when pre-load transformation suits a Snowflake warehouse and distinguish its trade-offs from ELT and ETLT.

How it connects

Contrasts ETL with ELT and ETLT, and connects it to extraction, staging, data warehouse, and presentation layers.

liveBookmanning.com

Read Snowflake Data Engineering on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Effective Data AnalysisETL is the process of extracting data from sources, transforming it into a suitable structure, and loading it into a target storage system.
• 

  liveBookmanning.com
  Delivering Machine Learning Projects: From design to deploymentETL is a data-integration process that extracts data from sources, transforms it, and loads it into a target system or data store.
• 

  liveBookmanning.com
  Grokking AI ApplicationsExtract, transform, load is a data-integration process that extracts data from sources, transforms it, and loads it into a target system.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionExtract, transform, load is a data-integration process that retrieves raw data, transforms it into a target structure, and loads the result for downstream use.
• 

  liveBookmanning.com
  Architecting an Apache Iceberg LakehouseA workflow that extracts data from sources, transforms it, and loads it into a target data system.

Compare how 6 books explain ETL →

Related concepts

• eltalternative to · ELT is a data-integration approach that extracts data, loads it into a target environment, and transforms it there.
• pipelineimplements · A pipeline is an ordered or coordinated sequence of connected processing stages or tasks that passes data, work, or results toward an outcome.
• datauses · Data is represented information, observations, measurements, or other values that can be stored, processed, transmitted, analyzed, or interpreted.
• Atlas Data Federationaddressed by · Atlas Data Federation is the MongoDB Atlas service for querying and combining data across multiple heterogeneous sources.
• data engineeringuses · Data engineering is the discipline of designing and operating systems and processes that collect, transform, integrate, store, and deliver data for computational use.
• etltused by · ETLT is a data pipeline pattern that performs simple transformations before loading and complex business transformations after loading.

This explainer was generated by Manning's knowledge graph from the text of Snowflake Data Engineering. The section links point to the parts of the book it draws on.