A stream ID identifies the CDC stream assigned to changes from a partition during a stream generation. ScyllaDB maps each partition key to one stream ID; multiple partitions can share it, but a partition maps consistently within that generation. section 12.1.2
Use stream IDs to understand and query CDC data:
- CDC partitions its log by stream ID and orders changes within each stream, so queries should target one stream ID at a time. section 12.1.2
- A topology change, such as adding a node, creates a new stream generation with its own set of stream IDs. section 12.1.2
- A stream ID does not identify one table row: several partition keys may map to the same ID. section 12.1.2
Querying the current CDC streams
A CDC export first reads the greatest timestamp from system_distributed.cdc_generation_timestamps, truncates it to milliseconds, retrieves the stream IDs from cdc_streams_descriptions_v2, and queries the CDC table for each returned ID. This accounts for the current generation and avoids relying on a full-table scan. section 12.1.2 section 12.3
