Partition
A partition is a distinct subdivision of a larger data, storage, or work unit that can be stored, managed, or processed separately.
Apache Kafka in Action
A Kafka partition is an ordered, append-only subdivision of a topic that stores records and enables parallel processing.
- How it is used
- Uses partitions to preserve related records’ local order while distributing topic storage and processing.
- How deep it goes
- Explains key-based placement, offsets, consumer assignment, replication, and partition files; also demonstrates creation and production with Kafka command-line tools.
- Examples
- Demonstrates product-price updates across two partitions using topic creation and keyed console-producer commands.
- What you can do after
- Operate/configure it with Kafka command-line tools and choose keys and partition counts with ordering and consumer parallelism in mind.
Kafka for Architects
A Kafka partition is an ordered, independently replicated subdivision of a topic that stores records and supports parallel processing.
- How it is used
- Divides Kafka topics into units for parallel processing while retaining order where related records share a partition.
- How deep it goes
- Explains partition indexes, append order, offsets, consumer assignments and scaling limits; defers partition-selection strategies and partition-count optimization.
- Examples
- Uses a customer-data service and diagrams of topic partitioning and parallel consumer assignments.
- What you can do after
- Apply it to plan consumer parallelism, preserve related-record order and recognize when partition counts limit scaling.
Kafka Streams in Action, Second Edition
A Kafka partition is an ordered, append-only subdivision of a topic that stores records and assigns each record a unique offset within that partition.
- How it is used
- Organizes topic storage and distributes records for broker throughput and load balancing.
- How deep it goes
- Explains partition directories, append behavior, offset indexes, sizing tradeoffs, and Kafka Streams task scaling.
- Examples
- Uses purchases storage directories and a two-partition console producer-consumer exercise with keyed records.
- What you can do after
- Operate/configure partition counts and inspect or consume records from specific topic partitions.
Read how Kafka Streams in Action, Second Edition explains partition →
Akka in Action, Second Edition
A partition is an ordered subdivision of a Kafka topic to which records are assigned and within which offsets identify record positions.
- How it is used
- Organizes Kafka topic records for consumer assignment and position tracking.
- How deep it goes
- Explains offsets, committed positions, consumer-group assignment, rebalancing, and failure reassignment.
- Examples
- Uses the test topic, test-0 assignment, offset reset to zero, and hello and hi records at offsets 0 and 1.
- What you can do after
- Operate Kafka consumers by reading assigned partitions, selecting starting offsets, and resuming from committed positions.
Read how Akka in Action, Second Edition explains partition →
Related concepts
- topicpart of · A topic is a named messaging destination or stream to which producers publish messages or records and from which consumers receive or read them.
- partitioning keyused by · A partitioning key is a value or attribute used to determine which partition stores or receives a record or data item.
- Apache Kafkapart of · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.
- consumer groupused by · A consumer group is a logical set of message consumers that cooperatively processes a shared stream by distributing its partitions among group members.
- log segmenthas part · A log segment is a file-backed portion of a Kafka partition log containing a contiguous range of records and associated indexes.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.