liveBook · Manning

ExecutionGraph

As explained in Kafka for Architects

An ExecutionGraph is Flink's distributed execution representation describing how a job's tasks run across the cluster.

An ExecutionGraph is Flink’s distributed representation of how a job’s tasks will run across a cluster. The JobManager creates it after receiving a JobGraph from the client. section 8.4.2

The client first turns a program’s sources, transformations, and sinks into a logical graph, then converts that graph into a JobGraph with task dependencies and execution optimizations. The JobManager uses the JobGraph to determine task parallelism, dependencies, and resource requirements, then deploys tasks to available TaskManagers. section 8.4.2

Figure 8.21 The Flink architecture. The Flink Client is responsible for transforming a program into a graph, which is submitted to the JobManager. JobManager converts it into an ExecutionGraph and deploys tasks to available TaskManagers.
A diagram of a task manager Description automatically generated

The architecture diagram traces the handoff from client-generated graph to JobManager-created ExecutionGraph and deployed tasks. section 8.4.2

How Kafka for Architects covers it

How deep it goes
Explains the graph’s place in job submission and its parallelism, dependency, and resource decisions, without detailing its internal data structure.
Examples
Uses a Flink architecture diagram showing the client-to-JobManager handoff and deployment to TaskManagers.
What you can do after
Evaluate/understand how Flink turns a submitted JobGraph into distributed task execution.
How it connects
Connects logical graphs, JobGraphs, the JobManager, and TaskManagers in Flink’s deployment flow.
Cover of Kafka for Architects
liveBookmanning.com
Read Kafka for Architects on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.