liveBook · Manning

Dataset split

As explained in Microsoft Azure in Action

A partition is a subset of database data stored separately from other subsets for distribution and access management.

A growing table or collection can become slow to search and awkward to manage. Partitioning divides its data into smaller subsets while, for a relational table, preserving the appearance of one table. A query that identifies the relevant subset can avoid scanning unrelated data. section 10.4.3

Separate recent and historical records

A bookstore with millions of books and orders could partition books by publication date and orders by order date. That separates older records from current activity, which may be accessed more often. Genre is another possible split for books; the useful choice depends on the queries people actually run. section 10.4.3

In Azure Cosmos DB, a partition key is an attribute used to decide where each item belongs. Cosmos DB distributes data across partitions as it grows, but choosing the key remains your job. A store might group products by category so category-based queries can locate related items quickly. The catch: a key that sends too much data or traffic to one place creates a hot partition. section 11.3.2 section 11.3.3

Choose the split with both distribution and everyday queries in mind:

  • Prefer a key with many distinct values when that helps spread data and requests evenly; a product type with only a few values may concentrate the load. section 11.3.2
  • Consider region when sales are geographically diverse, or a region–category combination when region alone distributes traffic unevenly. A time-based component can suit daily sales records. section 11.3.3
  • Make common queries filter on the partition key where practical. Queries spanning partitions can increase request-unit cost and latency. section 11.3.3
Listing 10.3 Partitioning the Books table by publication year
-- Example of partitioning the Books table by publication year
CREATE PARTITION FUNCTION pf_PublicationYear (int)
AS RANGE RIGHT FOR VALUES (2000, 2005, 2010, 2015, 2020);

CREATE PARTITION SCHEME ps_PublicationYear
AS PARTITION pf_PublicationYear
ALL TO ([PRIMARY]);

ALTER TABLE Books
ADD CONSTRAINT pk_Books PRIMARY KEY CLUSTERED (BookID)
ON ps_PublicationYear(PublicationYear);

The SQL example shows a publication-year partition function and scheme applied to a Books table. Its scheme assigns every partition to PRIMARY, so it illustrates the split without demonstrating placement across filegroups. section 10.4.3

C:/> az cosmosdb mongodb collection create 
--account-name alpacacosmosdbaccount --database-name alpacacosmosdb 
--name alpacacontainer --resource-group AlpacaRG --shard "/category" 
--throughput 400

The Azure CLI example creates a Cosmos DB MongoDB collection with category specified through --shard. section 11.3.2

How Microsoft Azure in Action covers it

How deep it goes
Defines relational partitions and explains partition-key choices, hot partitions, and cross-partition query trade-offs with configuration examples.
Examples
Uses bookstore publication and order dates, plus retail product categories and regions, to illustrate candidate splits.
What it assumes
Introduces partitions and filegroups, while its examples use tables, queries, data attributes, SQL, and Azure CLI commands.
What you can do after
Operate/configure it with SQL partition functions and schemes or Azure CLI --shard, while weighing uneven load against cross-partition queries.
How it connects
Connects partitioning to data modeling, filegroups, horizontal scaling, partition keys, and query performance.

Also explained in

Compare how 21 books explain dataset split →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Microsoft Azure in Action. The section links point to the parts of the book it draws on.