GlobalKTable

As explained in Kafka Streams in Action, Second Edition

GlobalKTable is a Kafka Streams abstraction that stores all records of an underlying topic on every application instance without partition sharding.

GlobalKTable broadcasts a complete update stream to every Kafka Streams application instance. Unlike a partitioned KTable, each instance stores every record from the source topic, making the table useful for reference data and stream enrichment. section 8.0 section 8.5

Use a GlobalKTable when the application needs to:

• Broadcast reasonably static information, such as user or sensor metadata, to every instance. section 8.5 section 8.6.3
• Join a KStream without co-partitioning, even when the stream key is null or does not match the table key. section 8.6 section 8.6.3
• Materialize the complete source topic locally in a KeyValueStore; the source topic provides recovery data instead of a generated changelog topic. section 8.5

Enriching unkeyed sensor events

Create the table with builder.globalTable("sensor-info", ...), then join sensor readings by extracting the sensor ID from each event: sensorKStream.join(sensorInfoGlobalKTable, (key, sensor) -> sensor.getId(), (sensor, info) -> formatReading(sensor, info)). The result retains the stream key, even when that key is null; a separate thread updates the GlobalKTable outside normal record processing.

Figure 8.13 GlobalKTable contains all records in a topic on each application instance.

Compare partitioned KTable ownership with the full GlobalKTable copy held by every application instance. section 8.5

How Kafka Streams in Action, Second Edition covers it

How deep it goes

Explains partition replication, local materialization, recovery, join key extraction, and dedicated update processing.

Examples

Uses user enrichment, website clickstreams, and unkeyed IoT sensor readings joined with sensor metadata.

What it assumes

Builds on KStream and KTable concepts, especially keys, partitions, update streams, and joins.

What you can do after

Apply KStream-GlobalKTable joins by deriving the lookup key from each stream record.

How it connects

Connects KStream joins, KTable partitioning, co-partitioning, KeyValueMapper, ValueJoiner, and reference tables.

liveBookmanning.com

Read Kafka Streams in Action, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Kafka for ArchitectsGlobalKTable is a Kafka Streams interface for a globally replicated, immutable, read-only table that is available to every application task.

Compare how 2 books explain GlobalKTable →

Related concepts

• KeyValueStoreuses · KeyValueStore is the Kafka Streams state-store interface for persistently storing and retrieving key-value records.
• KTablecontrasts with · KTable is a Kafka Streams abstraction representing the latest value associated with each key as a continuously updated table.
• stream-global-table joinused by · A stream-global-table join combines records from a KStream with matching records in a fully replicated GlobalKTable.
• changelogcontrasts with · A changelog topic is a Kafka topic that records state-store updates so processing state can be recovered or restored.
• topicuses · A topic is a named messaging destination or stream to which producers publish messages or records and from which consumers receive or read them.
• copartitioningcontrasts with · Copartitioning is the arrangement of input topics with compatible partitioning so records with corresponding keys can be processed together.

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.