liveBook · Manning

Message

As explained in Kafka for Architects

A message is a discrete data record transmitted between components through a messaging system.

A message is a Kafka record that carries event information between applications. A producer sends it to a broker-hosted topic; a consumer subscribes to the topic and pulls messages from it. The applications need not communicate directly. section 1.3.1 section 2.1

A message contains a key, a value, and optional headers. Topics group related messages, commonly by type; partitions organize the data within topics. “Event” usually describes a domain occurrence, while “message” often refers to the record that carries it. section 2.0 section 2.2

An address change

When a customer changes address, the service owning the data can publish a notification. Billing, analytics, and order services can each receive and process it independently. Business requirements determine which notifications to publish; the producer owns their structure and should make them reusable. section 1.4.1

On arrival, a message is appended to a log and stored durably. Messages cannot be individually changed or removed: correct bad data with a new message. Retention settings determine how long messages remain available, while a compacted topic retains at least the latest message for each key. Consumers poll for batches, process them, and commit offsets to record where to resume—not to prove processing succeeded. section 1.4.3 section 2.0 section 4.2

Figure 2.1 An illustration of how data is processed and stored in Kafka. Data is organized into topics, which are further divided into partitions. The basic units of information are messages, which contain keys, values, and headers.
A screenshot of a computer

AI-generated content may be incorrect.

Shows messages and their fields in the context of brokers, topics, and partitions. section 2.1

How Kafka for Architects covers it

How deep it goes
Defines message fields and follows storage and consumer polling, without specifying a message schema.
Examples
Illustrates address-change notifications for multiple subscribers and separate profile and transaction feeds for a customer dashboard.
What it assumes
Introduces topics as logical destinations while treating prior messaging experience as optional.
What you can do after
Evaluate/understand message grouping and retention, and distinguish an offset commit from proof of successful processing.
How it connects
Links messages to topics, partitions, commit logs, and retention policies.

Also explained in

Compare how 7 books explain message →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.