Kafka broker
A Kafka broker is a server process in an Apache Kafka cluster that stores records and handles producer, consumer, and other client requests.
Apache Kafka in Action
A Kafka broker is a server process that stores and serves Kafka records and participates in cluster operations.
- How it is used
- Serves as the cluster server that receives, stores, and makes partition records available to producers and consumers.
- How deep it goes
- Traces produce requests through network threads, queues, I/O threads, page cache, replication, and acknowledgments; treats complex production tuning as case-dependent.
- Examples
- Uses bank transfers and a three-broker topic inspection to illustrate record flow, replica placement, and replication-factor limits.
- What you can do after
- Operate/configure it: inspect partition leaders and replicas, choose a feasible replication factor, and provide resilient bootstrap addresses.
Kafka Streams in Action, Second Edition
A Kafka broker is a server process that stores Kafka records and handles requests from Kafka clients and consumers.
- How it is used
- Provides Kafka’s durable storage layer and serves requests for writing, reading, and obtaining cluster metadata.
- How deep it goes
- Explains storage, retention, replication, topics, partitions, request handling, and health checks, while deferring some client details.
- Examples
- Uses produce and fetch requests, partitioned topics, replicated broker clusters, Docker-based brokers, and JMX health metrics.
- What you can do after
- Operate a broker through Docker, send and consume records, and diagnose request queues or timeout-related performance issues.
Read how Kafka Streams in Action, Second Edition explains Kafka broker →
Kafka for Architects
A Kafka broker is a Kafka server that stores, receives, and serves records for producers and consumers without interpreting their application-level content.
- How it is used
- Handles producer and consumer traffic, record storage, replication, and stream-position storage within event-driven architectures.
- How deep it goes
- Explains cluster roles, opaque records, consumer polling, offsets, and broker limitations, but defers detailed schema and compatibility design.
- Examples
- Uses a broker cluster diagram, Schema Registry message flow, and consumer polling, processing, batching, and offset-commit workflow.
- What you can do after
- Operate/configure broker-based flows while placing deserialization, validation, routing, and processing in clients or intermediary services.
Related concepts
- transaction coordinatorincludes · A transaction coordinator is a component that coordinates distributed transaction participants and directs them to commit or roll back as a unit.
- binary searchuses · Binary search is an algorithm that locates a target or boundary in an ordered collection by repeatedly comparing with a midpoint and halving the remaining search range.
- serverkind of · A server is a computer system or software process that receives requests from clients or other systems and provides resources, data, or services in response.
- clientused by · A client is a person, organization, device, or software component that receives or uses a product, service, resource, or functionality provided by another entity.
- primary nodeincludes · A primary node is a designated authoritative or coordinating member of a distributed system, typically responsible for accepting operations or coordinating replicated state.
- Kafka clusterpart of · A Kafka cluster is a group of Kafka brokers that collectively store, distribute, and serve event streams.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.