liveBook · Manning

Continuous query

As explained in Kafka for Architects

A continuous query is a query that remains active and incrementally updates its results as new data arrives.

A continuous query remains active against an event stream, processing each arriving record and publishing changed results without rerunning the query. In ksqlDB, the stream is backed by a Kafka topic, while the output becomes another topic for downstream consumers. section 8.4.1

Continuous queries support several streaming patterns:

  • Filter high-value orders and publish matching events immediately. section 8.4.1
  • Maintain grouped counts, updating the result whenever a new order arrives. section 8.4.1
  • Restrict aggregations to time windows, such as one-hour tumbling intervals. section 8.4.1

Filtering expensive orders

A query such as CREATE STREAM Expensive_Orders AS SELECT * FROM Orders WHERE amount_paid > 100 EMIT CHANGES continuously emits qualifying orders. Downstream applications subscribe to the output topic instead of repeatedly querying a static result. section 8.4.1

Listing 8.4 Creating a stream of expensive orders with filtering
CREATE STREAM Expensive_Orders AS  #1
    SELECT *
    FROM Orders  #2
    WHERE amount_paid > 100
    EMIT CHANGES;  #3

Shows the compact KSQL form for creating a filtered stream with continuously emitted changes. section 8.4.1

How Kafka for Architects covers it

How deep it goes
Explains stream filtering, grouping, and windows, and sketches translation into Kafka Streams applications without low-level implementation details.
Examples
Uses expensive-order filtering, per-country counts, and one-hour tumbling-window counts.
What it assumes
Assumes familiarity with basic SQL constructs while defining stream-backed data structures and continuous processing.
What you can do after
Build it with ksqlDB by defining a stream, applying SQL transformations, and emitting results to an output topic.
How it connects
Connects SQL, Kafka topics, ksqlDB, Kafka Streams, streaming aggregation, and windowed processing.

Also explained in

Compare how 2 books explain continuous query →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.