A persistent query is a server-side ksqlDB computation that continually evaluates incoming Kafka records and writes its results to a Kafka topic. It is the durable, shareable part of a streaming application: downstream Kafka consumers can read the produced records for the topic’s configured retention period. section 11.3 section 11.7
Use a persistent query when analysis should keep running independently of one requesting client. Base streams and tables expose Kafka topics to ksqlDB; a subsequent CREATE STREAM AS SELECT or CREATE TABLE AS SELECT query derives results and materializes them into another topic. section 11.4 section 11.3
Choose among query types by where results go and how long evaluation lasts:
- Persistent queries continually run on the server and store results in a Kafka topic, making one computed result available to many consumers. section 11.3
- Push queries continually stream results only to the client that submitted them; they do not store those results in a topic. section 11.3
- Pull queries return a point-in-time result once and terminate; later updates require submitting another query. section 11.3
Count activities by last name
Create a materialized activity count from a user-activity stream: CREATE TABLE activity_count AS SELECT last_name, COUNT(activity) AS ACTIVITY_COUNT FROM user_activity GROUP BY last_name EMIT CHANGES; The resulting activity_count topic receives continuously updated counts. By default, the persistent query inherits the source key and value formats; here, a Protobuf-valued source produces a Protobuf-valued result and a corresponding Schema Registry schema. section 11.5
A persistent query can also serve as a format boundary. Add a WITH clause to override the output format when downstream clients need a different serialization, such as JSON instead of the source’s Protobuf format. ksqlDB registers schemas for persistent-query-derived streams or tables when applicable. section 11.5
Figure 11.1 Persistent queries run on the server and persist results to a topic.
Use the diagram to distinguish server-side, topic-backed computation from client-directed push results and one-time pull results. section 11.3
