liveBook · Manning

TopologyTestDriver

As explained in Kafka Streams in Action, Second Edition

TopologyTestDriver is a Kafka Streams test utility that runs a topology in memory and provides test input, output, and state-store access.

TopologyTestDriver runs a complete Kafka Streams topology in memory, letting tests exercise operators and state stores without a live Kafka broker. Use it alongside focused unit tests for individual mappers, aggregators, filters, and punctuators. section 14.1.3 section 14.2

A typical test follows this sequence:

  • Create the application and obtain the same Topology used by the running application. section 14.1.3
  • Construct a TopologyTestDriver, usually with try-with-resources, then create TestInputTopic and TestOutputTopic instances for the topology's topics. section 14.1.3
  • Pipe values, key-value pairs, TestRecord instances, or timestamped records into the input topic; read values, records, or key-value results from the output topic. section 14.1.3
  • Assert both emitted results and, for stateful topologies, the contents of named state stores retrieved through getKeyValueStore or related accessors. section 14.1.4

Testing a stateful score reducer

Create input and output topics, pipe scores for Anna, Matthias, and Neil, and assert that three output records preserve input order. Retrieve the named KeyValueStore and verify each score. After more inputs, compare the final output map with the store's latest values. For windowed or punctuated processing, supply Instants or advance the input topic's event time so stream-time behavior occurs deterministically. section 14.1.4 section 14.1.4

Listing 14.11 Building the TopologyTestDriver and input and output topics
try (TopologyTestDriver driver =
                 new TopologyTestDriver(yellingTopology)) {     #1

    TestInputTopic<String, String> inputTopic =      #2
            driver.createInputTopic("src-topic",
                                    stringSerializer,
                                    stringSerializer);

    TestOutputTopic<String, String> outputTopic =     #3
            driver.createOutputTopic("out-topic",
                                    stringDeserializer,
                                    stringDeserializer);

Inspect the compact setup showing driver construction and creation of serialized input and deserialized output topics. section 14.1.3

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Covers setup, record injection, output assertions, state-store inspection, and timestamp-driven behavior, while deferring broker integration to Testcontainers.
Examples
Uses a yelling topology, a stateful poker-score reducer, ordered records, state-store assertions, and timestamped window-processing records.
What it assumes
Assumes a Kafka Streams topology and serializers, while demonstrating test setup, topic harnesses, assertions, and stream-time control.
What you can do after
Diagnose/debug with it by piping deterministic records through a topology and comparing emitted results and state-store contents.
How it connects
Connects Kafka Streams topologies with TestInputTopic, TestOutputTopic, state stores, timestamps, windows, unit tests, and integration tests.

Also explained in

Compare how 2 books explain TopologyTestDriver →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.