Flink
Apache Flink is an open-source distributed processing framework for stateful computation over bounded and unbounded data streams.
Architecting an Apache Iceberg Lakehouse
Apache Flink is an open-source distributed processing framework for stateful batch and stream computations.
- How it is used
- Handles low-latency, stateful streaming and batch ingestion into Iceberg tables.
- How deep it goes
- Explains APIs, checkpoints, event time, transformations, and operational tradeoffs without descending into engine internals.
- Examples
- Uses Kafka-to-Iceberg SQL and DataStream examples alongside fintech, media, healthcare, and logistics ingestion scenarios.
- What you can do after
- Build real-time Kafka-to-Iceberg pipelines with transformations, checkpoints, event-time handling, and controlled commits.
Read how Architecting an Apache Iceberg Lakehouse explains Flink →
Kafka for Architects
Apache Flink is an open-source distributed stream-processing framework for stateful computations over bounded and unbounded data streams.
- How it is used
- Positions Flink as a processing layer that transforms and routes events, particularly when pipelines must also reach beyond Kafka.
- How deep it goes
- Defines jobs, stream bounds, and connector roles, then outlines runtime deployment without a low-level implementation walkthrough.
- Examples
- Illustrates content-based address routing and a Customer 360 operational data store, including a depicted design without Kafka.
- What you can do after
- Evaluate/understand when Flink’s batch-and-stream capabilities and external connectors suit a pipeline better than a Kafka-centered option.
Related concepts
- Apache Kafkauses · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.
- Kafka Streamsalternative to · Kafka Streams is a client library for building applications that continuously process data from and write data to Apache Kafka topics.
- checkpointinguses · A checkpoint is a designated point in a process at which its state or progress is recorded or evaluated, often to support recovery, continuation, or control.
- streamingused by · Streaming is the incremental handling of a sequence of data items, including their processing, transfer, or consumption as they become available.
- Apache Iceberguses · Apache Iceberg is an open table format for managing large analytical datasets with metadata, schema evolution, snapshots, and transactional operations across distributed or object storage.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.