chapter seven

7 Topics and partitions

 

This chapter covers:

  • Diving into topics including creation parameters and configuration options
  • Exploring how partitions exist as log files written on disk
  • Examining how segments impact the data inside partitions
  • Check out a way to test with EmbeddedKafkaCluster
  • Evaluating topic compaction and its impact on how data can be retained

In this chapter, we will look further into how we might store our data across topics as well as how to create and maintain topics. This will include how partitions fit into our design considerations and how we can view our data on the brokers. All of this information will help us as we look at how to make a topic show data like a database table that updates, rather than an append only log!

7.1 Topics

7.1.1 Topic Creation Options

7.1.2 Replication Factors

7.2 Partitions

7.2.1 Partition Location

7.2.2 Viewing Segments

7.3 More Topic and Partition Maintenance

7.3.1 Removing a Topic

7.3.2 Reelect Leaders

7.3.3 EmbeddedKafkaCluster

7.3.4 Kafka Testcontainers

7.4 Topic Compaction

7.4.1 Compaction Cleaning

7.4.2 Compaction Record Removal

7.5 Summary