Fine parallel processing

As explained in Kubernetes in Action, Second Edition

Fine parallel processing assigns multiple work items sequentially to each of a small number of pods that process items concurrently.

Fine parallel processing uses a small number of pods to take multiple work items from a queue. The pods process items concurrently until the queue is empty, avoiding one pod per item. section 18.1.5

Processing monthly quiz responses

Three pods process twelve monthly work items from MongoDB. After the queue is empty, all three pods complete and the Job reports completion.

Figure 18.10 The difference between coarse and fine parallel processing

Compare fine processing with coarse processing, where each pod handles one queue item and then terminates. section 18.1.5

How Kubernetes in Action, Second Edition covers it

How deep it goes

Explains queue retrieval, repeated worker loops, Job parallelism, and completion, but does not detail queue implementation beyond MongoDB.

Examples

Demonstrates three pods processing twelve monthly quiz-response items stored in a MongoDB collection.

What it assumes

Assumes containers implement both the queue and item retrieval, while demonstrating the Job and worker pattern.

What you can do after

Apply it via a Kubernetes Job and queue worker that repeatedly claims and processes items until none remain.

How it connects

Connects work queues with Kubernetes Jobs, pods, MongoDB, and coarse parallel processing.

liveBookmanning.com

Read Kubernetes in Action, Second Edition on liveBook Get the book at manning.com

Related concepts

• coarse parallel processingalternative to · Coarse parallel processing assigns one work item to each pod, with each pod terminating after processing that item.

This explainer was generated by Manning's knowledge graph from the text of Kubernetes in Action, Second Edition. The section links point to the parts of the book it draws on.