liveBook · Manning

Directory table

As explained in Snowflake Data Engineering

A directory table is a Snowflake table-like metadata structure that lists files and their properties for a stage.

A directory table is stage-associated metadata that lists files and their properties. It is not a separate stand-alone Snowflake object and can be enabled when creating or altering an internal or external stage. section 3.2

Use directory tables to inspect staged files beyond the information returned by LIST:

  • Query relative paths, file sizes, last-modified timestamps, URLs, MD5 hashes, and ETags. section 3.2
  • Refresh external-stage metadata automatically through cloud-provider event notifications, or refresh it manually when notifications are unavailable. section 3.2
  • Refresh internal-stage metadata manually; internal stages currently do not support automatic refresh. section 3.2
alter stage BISTRO_STAGE
set directory = (enable = true);

Enable a directory table on an existing external stage. section 3.2

alter stage BISTRO_STAGE refresh;

Synchronize directory metadata with the external-stage location when automatic event notification is not configured. section 3.2

Inspect staged files

Query the directory table with SELECT * FROM DIRECTORY(@BISTRO_STAGE); to inspect available files and their metadata. Sample results include dated CSV paths, sizes, modification timestamps, and ETags. section 3.2

How Snowflake Data Engineering covers it

How deep it goes
Defines creation, refresh, and querying workflows, but does not detail cloud event-notification implementation.
Examples
Uses BISTRO_STAGE, dated CSV files, ALTER STAGE, manual refresh, and DIRECTORY queries.
What it assumes
Assumes familiarity with Snowflake stages and staged files while explaining directory-table operations directly.
What you can do after
Operate directory metadata by enabling, refreshing, and querying it for staged-file properties.
How it connects
Connects stages, external stages, event notification, the LIST command, and the COPY command.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Snowflake Data Engineering. The section links point to the parts of the book it draws on.