An event notification is metadata indicating that a specified event occurred, such as a new file arriving in cloud storage. Snowpipe stores these notifications in a queue, identifies the new files, and loads them into Snowflake tables according to the pipe’s COPY statement. Notifications contain file metadata, not file contents; the files remain in cloud storage until ingestion. section 5.3 section 5.5
Cloud messaging implementations supported for continuous Snowpipe loading include:
- Amazon S3 uses Simple Queue Service notifications for bucket events. section 5.3
- Google Cloud Storage uses Pub/Sub messages for storage events. section 5.3
- Microsoft Azure blob storage uses Event Grid messages; the workflow includes a storage queue, event subscription, and system topic for blob events. section 5.3 section 5.5
Azure continuous ingestion
When a new file appears in an Azure blob-storage container, Event Grid produces a notification and places it in the message queue. Snowpipe consumes the notification and executes its COPY command to ingest the file. section 5.5
