A dataset split divides data into separately managed subsets. In distributed systems, sharding is horizontal partitioning: data is split across databases or nodes, usually by key, so capacity can scale horizontally. Partitioning also describes divisions made for manageability and parallelism; it is not the same as replication, which copies data across nodes. section 6.5
Interview question: how would you distribute growing account data?
Start by choosing a key that determines where each record belongs. Compare range-based and hash-based sharding, then explain how requests reach the right shard. Expect follow-ups about rebalancing, hotspots, cross-shard transactions, and resharding without downtime. The choice affects throughput and availability, but also consistency, latency, and operational complexity. section 6.5
The same split can reduce contention in a cache. Sharded caches divide the keyspace among in-memory instances, each with its own eviction logic. A shard can use Redis as a fallback after a miss or cold start; pre-warming common keys can reduce initial response times. section 6.4
Table 6.6 Data replication, sharding, and partitioning
|
Topic
|
Fundamentals
|
Applied skills
|
Advanced
|
|
Data replication
|
Understand that data is copied across nodes to improve availability and fault tolerance.
|
Use leader-follower or multi-leader replication. Tune sync vs. async replication. Handle stale reads and conflicts.
|
Optimize replication lag, implement quorum reads/writes, and reason about consistency in geo-replicated systems.
|
|
Sharding (horizontally partitioning)
|
Understand that data is split across multiple databases or nodes, usually by key.
|
Implement horizontal scaling using range-based or hash-based sharding. Handle rebalancing and routing logic.
|
Design for cross-shard transactions, hotspot detection, and dynamic resharding without downtime.
|
|
Partitioning
|
Know that partitioning divides data into independent subsets for manageability and parallelism.
|
Apply vertical vs. horizontal partitioning. Use partitioning to reduce query latency and contention.
|
Design partition-aware data models and query strategies. Handle skew, repartitioning, and partition pruning at scale.
|
Compare replication, sharding, and partitioning across fundamental, applied, and advanced design questions. section 6.5























