A row group is a Parquet storage and I/O unit containing a subset of a columnar file’s rows. Within it, column chunks let query engines read only requested columns, while footer statistics help skip irrelevant groups through predicate pushdown. section 5.3.1
Row-group design creates several practical tradeoffs:
- Smaller row groups can improve parallelism and reduce memory pressure, while oversized groups can cause execution spills because groups are not splittable. section 10.2.2
- File size and row-group size are related but distinct: files are splittable, whereas parallelism is driven by row-group size. section 10.2.2
- The appropriate size depends on query-engine behavior, compression, table width, nested data, executor memory, and I/O bandwidth. section 10.2.2
Figure 5.8 Using metadata in Parquet row-group footers, we can tell which row groups to scan and which to skip for faster reads.
Inspect how Parquet row-group footer metadata identifies groups to scan or skip during query planning. section 5.3.1
