Partition evolution lets an Apache Iceberg table change its partitioning strategy as data volumes, business requirements, and query patterns change, without rewriting historical data. It avoids the rigidity of fixed partition schemes, which can otherwise require data migration, duplication, or complex workarounds. section 2.4 section 2.6.2
The technique supports long-lived datasets whose access patterns mature over time. It works alongside hidden partitioning, which keeps transform-based partition columns out of user-facing queries, and with sort orders and compaction, which improve physical data locality when partitioning alone is insufficient. section 1.6 section 10.1.2
Adapting a growing table
A table may begin with a partition scheme that suits its initial workload, then adopt a more appropriate scheme as data characteristics or query patterns evolve. New organization can improve colocation for later data while leaving historical files intact. section 10.1.2
Figure 10.3 With better colocation of data, a query for all “yellow” records needs to read only one file instead of all three. Partitioning allows records with unique values to be written to distinct data files.
Use the colocation illustration to see how partitioning can place records in distinct files and reduce unnecessary data reads. section 10.1.2
