liveBook · Manning

Queuing

As explained in Snowflake Data Engineering

Query queuing is the waiting of queries for warehouse resources when the warehouse cannot execute them simultaneously.

Query queuing occurs when a virtual warehouse lacks sufficient resources to execute all incoming queries simultaneously. Waiting increases query execution time and can reduce performance. section 9.4

Shared warehouses are more likely to queue queries when many users or applications submit work concurrently. Reducing waiting time supports the balance between pipeline performance and compute cost. section 9.0 section 9.4

Use these approaches to reduce queuing:

  • Create additional warehouses and assign different warehouses to different user or application groups when a single-cluster warehouse is overloaded. section 9.6
  • Convert a single-cluster warehouse to a multicluster warehouse when concurrent workloads need more capacity. section 9.6
  • Increase the maximum number of clusters in a multicluster warehouse when existing cluster capacity does not absorb concurrent queries. section 9.6
use role ACCOUNTADMIN;
grant database role SNOWFLAKE.USAGE_VIEWER to role SYSADMIN;
use role SYSADMIN;

Grant the SNOWFLAKE.USAGE_VIEWER database role to SYSADMIN before querying warehouse load history. section 9.4.1

select 
  to_date(start_time) as start_date, 
  warehouse_name, 
  sum(avg_running) as total_running, 
  sum(avg_queued_load) as total_queued
from SNOWFLAKE.ACCOUNT_USAGE.WAREHOUSE_LOAD_HISTORY
where TO_DATE(start_time) > DATEADD(day,-7,TO_DATE(CURRENT_TIMESTAMP()))
group by all
order by 1, 2;

Summarize AVG_RUNNING and AVG_QUEUED_LOAD by warehouse and day for the previous seven days. section 9.4.1

Interpreting warehouse load

WAREHOUSE_LOAD_HISTORY records workload in five-minute intervals. AVG_RUNNING represents the ratio of running query time to interval time, while AVG_QUEUED_LOAD represents the corresponding queued-time ratio. A trial-account report may show zero queuing because users are not querying warehouses simultaneously; production workloads can differ. section 9.4.1

How Snowflake Data Engineering covers it

How deep it goes
Defines contention, measures queue load, and recommends warehouse-scaling options without detailing cluster-selection internals.
Examples
Uses Snowsight warehouse views, WAREHOUSE_LOAD_HISTORY queries, and a bakery warehouse report grouped by day.
What it assumes
Requires familiarity with Snowflake warehouses, roles, database roles, and querying the SNOWFLAKE database.
What you can do after
Diagnose queuing with warehouse load metrics and choose additional warehouses or multicluster capacity to reduce it.
How it connects
Connects virtual warehouses, multicluster warehouses, concurrent queries, warehouse load history, and compute cost monitoring.

Also explained in

Compare how 2 books explain queuing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Snowflake Data Engineering. The section links point to the parts of the book it draws on.