Log segment
A log segment is a file-backed portion of a Kafka partition log containing a contiguous range of records and associated indexes.
Apache Kafka in Action
A log segment is a file containing a contiguous portion of the records in a Kafka partition log.
- How it is used
- Organizes a growing Kafka partition log into files that brokers append to and can later clean up.
- How deep it goes
- Explains segment files, offset-based names, inspection and rolling controls; defers the fuller treatment of cleanup.
- Examples
- Uses kafka-dump-log.sh output, a segment.ms configuration command and a partition-directory listing with successive .log files.
- What you can do after
- Operate/configure it by inspecting segment files, adjusting topic rolling limits and producing a message to trigger a roll check.
Kafka Streams in Action, Second Edition
A Kafka log segment is a file-backed portion of a partition's append-only log that contains a contiguous range of records along with its indexes.
- How it is used
- Uses segments to bound partition files, serve offset-based reads, and apply retention without scanning one monolithic log.
- How deep it goes
- Explains rolling triggers, directory files, relative offsets, binary search, and segment deletion, but not segment implementation code.
- Examples
- Uses purchases-0 directory listings, numbered segment files, and active-segment rollover to illustrate storage.
- What you can do after
- Evaluate/understand how Kafka locates records, rolls partition files, and removes segments under retention settings.
Read how Kafka Streams in Action, Second Edition explains log segment →
Related concepts
- segment roll interval settingconfigured by · A segment roll interval setting is a Kafka configuration specifying the maximum time a log segment remains active before a new segment is created.
- partitionpart of · A partition is a distinct subdivision of a larger data, storage, or work unit that can be stored, managed, or processed separately.
- log compactioncontrasts with · Log compaction is a storage technique that retains the latest record for each key while removing superseded records.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.