A backing topic is the Kafka topic from which a ksqlDB stream or table reads its underlying records. A base stream or table connects directly to that topic through its CREATE statement, defining the value columns and, when present, the record key. The topic’s key affects partitioning; without a key, records are evenly distributed across partitions. section 11.4
When mapping a backing topic into ksqlDB:
- Declare a stream key with KEY when the Kafka records contain one; declare a table key with PRIMARY KEY. section 11.4
- Specify the key and value formats, such as KAFKA for basic Kafka key types and JSON for values. section 11.4
- Use Schema Registry schemas to let ksqlDB infer column names and types instead of listing them in the CREATE statement. section 11.4
The user_activity topic
A stream can map JSON values from the user_activity backing topic with four partitions and event_time as its timestamp. If records carry integer user IDs, add user_id INT KEY and key_format='KAFKA'. A table over the same topic uses user_id INT PRIMARY KEY. section 11.4
Listing 11.22 Creating the user_activity stream
CREATE STREAM user_activity (first_name VARCHAR,
last_name VARCHAR,
activity VARCHAR,
event_time VARCHAR,
steps INT
) WITH (kafka_topic='user_activity',
partitions=4,
value_format='JSON',
timestamp = 'event_time',
timestamp_format = 'yyyy-MM-dd HH:mm:ss'
);
Inspect the complete CREATE STREAM statement for mapping user_activity values, partitions, JSON encoding, and event timestamps. section 11.4
