Checkpointing persists processing progress and state so a stream processor can resume consistently after interruption. In Atlas Stream Processing, checkpoint documents act as bookmarks that prevent missed or duplicated data. section 15.7.1 section 15.9
A stream processor commits a checkpoint through a defined sequence:
- A checkpoint document follows the processor’s logic flow and reaches the final operator. section 15.7.1
- The processor commits the checkpoint ID together with status records for each stateful operation. section 15.7.1
- After a restart, the processor retrieves the last committed checkpoint and resumes from its saved state. section 15.7.1
Checkpointing trades retained recovery information for storage. In WiredTiger, durable checkpoints provide consistent recovery points, while journaling records modifications made between checkpoints and replays them after an unexpected exit. section 8.1.1 section 8.1.2
Restarting a stateful stream processor
A processor aggregates events across stateful operations and commits a checkpoint after the final operator completes. If the processor stops, it loads the latest committed checkpoint, restores operator state, and continues from the recorded stream position rather than reprocessing an uncertain range. section 15.7.1
