6 Executing code natively with Snowpark

 

This chapter covers

  • The Snowpark architecture
  • Creating functions and stored procedures in Python worksheets
  • Using the SQL API from a local development environment
  • Transforming data with data frames
  • Deploying functions and stored procedures

Data engineers spend significant time developing data ingestion, transformation, and presentation applications. In addition to using SQL, Snowflake’s built-in querying language, data engineers can build applications in one of the supported programming languages, including Python, Java, and Scala, that run natively in Snowflake. Snowpark makes this possible, as described in more detail in this chapter.

Snowpark is an umbrella term encompassing features on the server and the client sides. These features are as follows:

  • On the server side, Snowpark executes code natively in Python, Java, and Scala. Data engineers don’t have to extract data from Snowflake and save it in their local development environment to work with the data. Instead, they can write code that executes inside the Snowflake processing engine.
  • On the client side, Snowpark provides API libraries for each supported programming language. These libraries contain functionality to connect to Snowflake, query data in Snowflake tables, transform the data, and deploy functions and stored procedures to the Snowflake server.

6.1 Introducing Snowpark

6.2 Creating a Snowpark procedure in a worksheet

6.3 Using the SQL API from a local development environment

6.3.1 Installing and configuring the local development environment

6.3.2 Creating a Snowflake session

6.3.3 Providing credentials in a configuration file

6.3.4 Querying data and executing SQL commands

6.4 Generating a date dimension in Snowpark Python

6.5 Working with data frames

6.6 Ingesting data from a CSV file into a Snowflake table

6.7 Transforming data with data frames

Summary