Segment roll interval setting

As explained in Apache Kafka in Action

log.roll.ms is a Kafka broker configuration that specifies the default time interval for rolling log segments.

The broker setting log.roll.ms defines the default maximum age of a Kafka log segment before Kafka creates a new one. Topics can override this default with segment.ms; Kafka also rolls a segment when it exceeds segment.bytes, 1 GiB by default. section 8.3.4

Rolling segments regularly makes retention more precise because Kafka deletes whole segments. A message can therefore remain longer than retention.ms while its segment continues receiving messages; segment.ms and retention.ms must be chosen together, with the retention-check interval also considered. section 10.3.1

Force a short roll interval for testing

Set a topic-specific interval of 60 seconds, produce messages, and inspect the partition directory. New log, index, time-index, and snapshot files show that Kafka created another segment. The broker-wide equivalent is log.roll.ms; the example uses segment.ms so it affects only the selected topic. section 8.3.4

Figure 8.4 Topics are divided into partitions. Partitions create segments as messages are produced. Segments consist of the actual log files, index files, and a snapshot file.

Trace how produced messages become partition segments containing log, index, and snapshot files. section 8.3.4

How Apache Kafka in Action covers it

How deep it goes

Explains time- and size-based rolling, topic overrides, generated files, and retention trade-offs without detailing broker internals.

Examples

Demonstrates a 60-second topic override, message production, and partition-directory inspection showing newly created segment files.

What it assumes

Assumes basic Kafka topics and partitions while establishing how messages, segments, indexes, and retention relate.

What you can do after

Operate/configure Kafka by setting a topic interval, producing messages, and verifying segment creation on disk.

How it connects

Connects log.roll.ms with segment.ms, segment.bytes, retention.ms, and log.retention.check.interval.ms.

liveBookmanning.com

Read Apache Kafka in Action on liveBook Get the book at manning.com

Related concepts

• log segmentconfigures · A log segment is a file-backed portion of a Kafka partition log containing a contiguous range of records and associated indexes.
• segmentconfigures · A segment is a distinct portion or bounded region of a larger structure, system, or sequence, with boundaries or properties determined by its context.
• log retentionconfigures · Log retention is a policy governing how long log records are kept and when they are removed.

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.