Partition imbalance occurs when data-processing work is unevenly distributed across partitions. One partition might handle most incoming requests while others sit mostly idle. The overloaded partition can raise latency and limit throughput and scalability, even when the system has spare capacity elsewhere. section 5.5
Check two distinct sources of imbalance:
- Data distribution: A low-cardinality partition key offers few values across which to spread data. If those values are also unevenly distributed, some partitions can be much larger. Skewed writes can enlarge them further. section 5.2.1
- Access patterns: Requests can concentrate on a few hot partitions even when data is evenly distributed. Demand can also shift over time, making a previously balanced layout insufficient. section 5.5 section 5.5.2
An online store
User IDs offer many distinct partition-key values; order statuses such as pending and shipped offer only a few. Even with well-distributed product data, a launch can send many requests to the partition holding one new product. A sale can create a similar temporary hotspot. section 5.2.1 section 5.5.2
Start by examining the proposed key’s cardinality and the distribution of its values, then consider which partitions receive requests and writes. A high-cardinality key gives data more opportunities to spread out, but does not guarantee balanced traffic. For hot partitions, consider partitioning within them or assigning them more resources. Rebalance partitions that have grown disproportionately; for predictable demand spikes, consider overprovisioning, load balancing, or scaling partitions. The choice depends on whether the imbalance comes from stored data or changing demand. section 5.2.1 section 5.5.2
