liveBook · Manning

Schema

As explained in Snowflake Data Engineering

A data schema is the structural specification of the fields, types, and organization of a dataset.

A schema specifies a dataset’s fields and data types before structured data is loaded. In Snowpark, StructType and StructField define that contract for CSV input. section 6.6

A practical schema records:

  • Field names such as Customer, Order_date, Delivery_date, Baked_good_type, and Quantity. section 6.6
  • Field types including StringType, DateType, and DecimalType. section 6.6
  • The mapping used when reading staged data into a data frame and copying it into a staging table. section 6.6

Defining and applying a CSV schema

Create a StructType containing one StructField per CSV column, then pass it to session.read.schema(schema_for_csv).csv("@orders_stage") before copying the data into ORDERS_STG.

Schemas are explicit for structured CSV data but are not required in the same way for semistructured JSON. Snowflake can store changing or nested JSON in VARIANT before flattening it into relational columns. section 4.1.4

How Snowflake Data Engineering covers it

How deep it goes
Explains field-level Python definitions and their application during loading, while contrasting them with schema-flexible JSON ingestion.
Examples
Uses an orders CSV schema with customer, dates, baked-good type, and quantity fields, then contrasts it with nested JSON stored in VARIANT.
What it assumes
Assumes basic Snowpark Python and staged-file knowledge while teaching the required StructType, StructField, and data-type definitions.
What you can do after
Build it from scratch by defining Snowpark fields and types, applying the schema to staged CSV data, and loading a staging table.
How it connects
Connects schemas with Snowpark, CSV, data types, data frames, staging tables, VARIANT, and semistructured data.

Also explained in

Compare how 44 books explain schema →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Snowflake Data Engineering. The section links point to the parts of the book it draws on.