liveBook · Manning

Caching

As explained in Snowflake Data Engineering

Data caching is the retention of previously accessed data in faster storage or memory so subsequent operations can retrieve it more efficiently.

Data caching stores copies of previously retrieved or computed data in faster temporary storage, allowing Snowflake to reuse it instead of accessing original storage or re-executing work. section 9.3

Snowflake places caches across its cloud services and query processing layers:

  • The query results cache stores executed query results, usually for 24 hours, so repeated queries can retrieve persisted results. section 9.3
  • The metadata cache stores database-object metadata, including table row counts and other properties; metadata-only queries can execute without reading table data or using a virtual warehouse. section 9.3
  • Each virtual warehouse has a warehouse cache containing data read from underlying tables; subsequent queries may read from it instead of database storage. section 9.3
  • Warehouse size determines warehouse-cache size, so larger warehouses provide larger cache areas. section 9.3

Use the query profile when investigating performance: data caching appears alongside micro-partition pruning and data spilling as a significant query-performance scenario. section 8.2.3

Figure 9.7 The three layers of the Snowflake architecture are database storage, query processing, and cloud services. The cloud services layer holds the metadata and the query results cache. The query processing layer holds the warehouse cache for each virtual warehouse.
figure

Architecture figure showing database storage, query processing, cloud services, metadata caching, query-results caching, and per-warehouse caching. section 9.3

How Snowflake Data Engineering covers it

How deep it goes
Defines caching and maps three cache types to Snowflake architecture layers, focusing on behavior rather than cache-management internals.
Examples
Uses repeated queries, metadata-only queries, and subsequent queries against the same virtual warehouse as caching examples.
What it assumes
Assumes familiarity with Snowflake’s cloud services, query processing, database storage, and virtual warehouse layers.
What you can do after
Evaluate/understand Snowflake query-performance behavior by distinguishing cache types and locating caching in the query profile.
How it connects
Connects data caching with Snowflake architecture, query results, metadata, virtual warehouses, query profiles, and data spilling.

Also explained in

Compare how 38 books explain caching →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Snowflake Data Engineering. The section links point to the parts of the book it draws on.