A cloud columnar cache is a local, disk-based cache on each executor node. It stores frequently accessed cloud data blocks in compressed columnar form, allowing subsequent queries to read locally instead of repeatedly fetching data from remote object storage. section 8.3.2 section 8.3.4
The cache improves federated-query operations by:
- Reducing cloud storage reads and query latency for repeatedly accessed data. section 8.3.4
- Lowering egress costs by avoiding repeated downloads of large files. section 8.3.4
- Managing stored data automatically with a Least Recently Used policy. section 8.3.4
Figure 8.7 Dremio can manage multiple coordinators to parse and plan query execution, and then distribute execution among multiple executors available to those coordinators, enabling workload scalability and availability.
The cache resides on executor nodes within an architecture that distributes planned query execution across multiple coordinators and executors. section 8.3.2
