liveBook · Manning

Partitioning key

As explained in Latency

A partitioning key is the data field or value used to determine which partition stores a record.

A partitioning key is the column whose value determines where a row goes when a table is horizontally partitioned. The rows remain part of one logical table, even though they are stored in separate partitions. In some distributed SQL databases, defining the key is the application developer’s main partitioning task; the database handles the split. section 5.2.1

Partition users by location

Consider a Users table with ID, Username, and Location columns. Using Location as the partitioning key places two New York users in one partition, one Cape Town user in another, and one Sydney user in a third. A query against the partitioned table must first identify which partitions it affects. section 5.2.1

Figure 5.1 Horizontally partitioning a database table. On the left, you can see the full logical table with four rows. On the right side, you can see three horizontal partitions.
Figure 5.1: Example of horizontal partitioning of a database table. At the left side of the diagram, you see the full logical table with four rows. On the right side of the diagram, you see three horizontal partitions.

Compare the four-row logical table with its three location-based partitions to see how one key value groups complete rows. section 5.2.1

Key choice affects balance. Suppose user ZIP codes are the key in a key-range scheme. If many users live in a few areas, some ranges may hold far more keys than others, creating hot partitions. Changing the key or switching to key-hash partitioning can improve distribution. Equal key counts are not sufficient, though: partitions can still differ greatly in size when their values differ, as with large images in an object store. Choose with the data distribution in mind; correcting a poor scheme may require rebalancing every partition on a live system. section 5.5.1

How Latency covers it

How deep it goes
Defines the key through a worked table example and examines distribution failures, but gives no key-selection procedure.
Examples
Uses a location-partitioned Users table, unevenly distributed ZIP codes, and differently sized images.
What it assumes
Assumes familiarity with table columns and SQL queries while illustrating horizontal partitioning with a small Users table.
What you can do after
Evaluate/understand how a candidate key distributes records and when a key or scheme change may address hot partitions.
How it connects
Connects horizontal partitioning and distributed SQL to query routing, key-range partitioning, key-hash partitioning, and hot partitions.

Also explained in

Compare how 10 books explain partitioning key →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Latency. The section links point to the parts of the book it draws on.