Checkpointing
A checkpoint is a designated point in a process at which its state or progress is recorded or evaluated, often to support recovery, continuation, or control.
Architecting an Apache Iceberg Lakehouse
A checkpoint is persisted progress information from which an incremental data-processing workflow can resume after interruption.
- How it is used
- Supports resumable, fault-tolerant ingestion across micro-batch and continuous streaming workflows.
- How deep it goes
- Explains offsets, state, retries, and transactional coordination, while deferring implementation-specific configuration and recovery tuning.
- Examples
- Uses Spark Kafka-to-Iceberg streaming, Flink checkpointed Kafka ingestion, and five-minute micro-batch progress boundaries.
- What you can do after
- Operate and recover incremental ingestion pipelines from persisted progress while limiting duplicates, skipped records, and partial commits.
Read how Architecting an Apache Iceberg Lakehouse explains checkpointing →
Kafka for Architects
Checkpointing is the periodic capture and persistence of streaming application state so it can be restored after failure.
- How it is used
- Protects stateful Flink streaming jobs against failures by periodically capturing application state.
- How deep it goes
- Contrasts memory-held checkpoints with external storage and their failure implications, without detailing configuration or recovery steps.
- Examples
- Uses orders grouped by region and a source operator’s reading progress to illustrate state worth retaining.
- What you can do after
- Operate/configure it by selecting durable external checkpoint storage instead of relying on JobManager memory for production recovery.
ScyllaDB in Action
Checkpointing records completed progress so that a process can pause and resume from an earlier point without restarting.
- How it is used
- Supports pausing and resuming bulk data migrations without repeating completed work.
- How deep it goes
- Explains the operational rationale and a simple storage approach, but stops short of defining a sophisticated checkpoint protocol.
- Examples
- Uses Spark Migrator progress tracking and custom migrator tables storing completed token ranges.
- What you can do after
- Operate a bulk migration that pauses and resumes from recorded progress.
AI Agents and Applications
- How it is used
- Preserves conversational context across turns in a LangGraph assistant, supporting follow-up questions and stateful workflows.
- How deep it goes
- Defines graph-state snapshots at super-steps and sketches recovery and human approval, then walks through conversational persistence setup.
- Examples
- Uses a router-based travel assistant, a shared thread ID, and InMemorySaver to demonstrate persisted chat turns.
- What you can do after
- Apply LangGraph checkpointing by assigning a thread ID and compiling a graph with a checkpointer to retain conversational state across turns.
Read how AI Agents and Applications explains checkpointing →
MongoDB 8.0 in Action, Third Edition
A stream processing checkpoint is a persisted record of processing progress and state that allows a stream processor to resume consistently after interruption.
- How it is used
- Maintains resumable progress and state for stream processors after interruption.
- How deep it goes
- Explains checkpoint records, commitment, restart recovery, and durable storage, but does not specify checkpoint configuration commands.
- Examples
- Uses checkpoint documents, stateful operators, processor restarts, WiredTiger data files, and journal replay as concrete examples.
- What you can do after
- Operate Atlas Stream Processing with resumable progress and understand how WiredTiger combines checkpoints with journaling for recovery.
Read how MongoDB 8.0 in Action, Third Edition explains checkpointing →
Building Reliable AI Systems
The practice of saving execution state so a process can be paused and later resumed.
- How it is used
- Supports agent state management by allowing execution to pause, resume, or recover after sandbox failure.
- How deep it goes
- Connects checkpointing to pause, resume, and crash recovery, but does not detail checkpoint formats or save-and-restore procedures.
- Examples
- Uses a sandbox crash as the recovery case: restore harness state and continue from the last checkpoint.
- What you can do after
- Understand how an agent harness can recover execution after a sandbox crash using the last checkpoint.
Read how Building Reliable AI Systems explains checkpointing →
Related concepts
- Flinkused by · Apache Flink is an open-source distributed processing framework for stateful computation over bounded and unbounded data streams.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.