liveBook · Manning

Parallelism

Also called parallelization, parallel execution, parallel processing, parallel computing, parallel computation

Parallelism is the execution of multiple independent tasks or computational operations at the same time rather than sequentially.

How 55 Manning books explain it

Fast Python: High performance techniques for large datasets

How it is used
Uses parallelism to accelerate Python data processing on multicore CPUs, GPUs, and distributed systems.
How deep it goes
Develops multiprocessing and Cython approaches, examines GPU and task-graph execution, and defers general parallel programming techniques.
Examples
Uses word-count MapReduce, a colored Game of Life grid, and partitioned dataframe maximums.
What you can do after
Apply it via Python multiprocessing, Cython’s OpenMP-backed prange, and Dask while assessing communication and serial bottlenecks.

Read how Fast Python: High performance techniques for large datasets explains parallelism →

Grokking Concurrency

Parallelism is the simultaneous execution or simultaneous progress of multiple computational operations.

How it is used
Introduces parallelism as simultaneous execution for speeding up decomposable work, contrasting it with serial execution and concurrency.
How deep it goes
Explains hardware and task-independence requirements, synchronization and scaling limits, while deferring detailed decomposition techniques.
Examples
Uses laundry loads, card sorting and a password-search implementation to illustrate simultaneous work and its limits.
What you can do after
Evaluate whether independent tasks and available processing resources make parallelism promising, and recognize serial and I/O bottlenecks.

Read how Grokking Concurrency explains parallelism →

CUDA for LLMs

Parallel processing is the concurrent execution of multiple computations across processing resources.

How it is used
Uses parallelism to map deep-learning operations onto GPU threads and guide kernel work distribution.
How deep it goes
Traces thread indexing, launch geometry, reductions, memory access, and block-level work partitioning from naive kernels to optimized strategies.
Examples
Works through 1000 × 500 matrix addition, naive GEMM, coalesced GEMV, and Flash Attention row ownership.
What you can do after
Build parallel CUDA kernels from scratch by assigning work to threads, sizing grids, and handling boundaries.

Read how CUDA for LLMs explains parallelism →

Latency

Parallelism is the simultaneous execution of tasks across multiple processors or other execution units.

How it is used
Uses parallelism to reduce application latency and improve throughput by dividing work across execution units.
How deep it goes
Explains task and data parallelism, speedup limits, coordination costs, and processor placement with worked performance examples.
Examples
Uses order processing, parallel product-page requests, and a Rust benchmark of sequential versus concurrent simulated waits.
What you can do after
Evaluate parallelization opportunities and speedup limits while accounting for dependencies, contention, and tail latency.

Read how Latency explains parallelism →

Functional Programming in Scala, Second Edition

Parallelism is the execution of multiple parts of a computation at the same time, typically on multiple processor cores or CPUs.

How it is used
Models parallel and asynchronous computations as composable Par values instead of directly coordinating threads.
How deep it goes
Derives Par, map2 strictness, laws, and generalized combinators, while deferring execution behind a separate interpreter.
Examples
Uses divide-and-conquer integer summation, conditional computation choice, and balanced string concatenation folds.
What you can do after
Build it from scratch as a composable Par library with combinators, laws, and a separate execution interpreter.

Read how Functional Programming in Scala, Second Edition explains parallelism →

Starting Data Analytics with Generative AI and Python

Parallelized code is software structured so that multiple portions of a computation can execute concurrently.

How it is used
Uses parallelism as a scaling option for data analytics and as a measure of simultaneous system workloads.
How deep it goes
Demonstrates threaded bootstrap code and performance trade-offs, while deferring distributed synchronization and fault-tolerance internals.
Examples
Compares optimized and threaded bootstrap confidence intervals and identifies independent Euclidean distance calculations as a suitable parallel workload.
What you can do after
Apply it via Python’s concurrent.futures and evaluate execution time against an optimized baseline.

Read how Starting Data Analytics with Generative AI and Python explains parallelism →

Advanced Algorithms and Data Structures

Parallel computing is the execution of multiple computational operations concurrently, typically using multiple processing units.

How it is used
Tests whether sequential clustering work can be split across workers to reduce running time.
How deep it goes
Works through k-means steps and ideal re-centering operation counts, then identifies result aggregation as a bottleneck.
Examples
Uses per-cluster k-means re-centering and coin counting to show independent work and the cost of combining results.
What you can do after
Evaluate/understand it by identifying independent algorithm steps and checking whether aggregation will limit the speedup.

Read how Advanced Algorithms and Data Structures explains parallelism →

Graph Neural Networks in Action

Parallel computing is a technique that divides work among multiple processing units so computations can execute concurrently.

How it is used
Uses parallelism to reduce GNN training time by spreading work across processors or machines.
How deep it goes
Compares two PyTorch parallelization options and their overhead, while stopping short of a configuration walkthrough.
Examples
Compares CPU, GPU, TPU, NPU, and IPU workloads and cites NVIDIA GPUs used on Colab.
What you can do after
Evaluate/understand when DistributedDataParallel may suit multi-GPU GNN training better than DataParallel.

Read how Graph Neural Networks in Action explains parallelism →

Vibe Engineering

Parallel execution is the concurrent performance of independent or separable computational tasks.

How it is used
Evaluates parallel processing and concurrent agent workflows as ways to improve throughput without assuming that more parallelism is faster.
How deep it goes
Explains work-stealing and agent coordination, then compares implementation overheads, benchmarks, and operational failure modes.
Examples
Compares account-lookup strategies and follows concurrent agent work across Git worktrees and an IDE.
What you can do after
Evaluate parallel execution against workload, coordination costs, benchmark results, and operational visibility.

Read how Vibe Engineering explains parallelism →

Grokking GPU Programming

Parallel computing is the execution of computational work simultaneously across multiple processing units.

How it is used
Introduces parallel thinking for dividing computational work among processing units, with GPU data parallelism as the main application.
How deep it goes
Defines parallelism and dependencies, works through array doubling and speedup limits, and defers performance techniques.
Examples
Uses array doubling, tournament-style summation, Fibonacci, and a program with 10% sequential work to illustrate parallel opportunities and limits.
What you can do after
Evaluate potential parallel work by identifying dependencies and recognizing how sequential portions limit speedup.

Read how Grokking GPU Programming explains parallelism →

100 Go Mistakes and How to Avoid Them

Parallel execution is the simultaneous execution of multiple instances of a task, typically on separate processing cores.

How it is used
Distinguishes simultaneous execution from concurrent structure and applies it to sorting and test execution.
How deep it goes
Explains scheduling and goroutine overhead, then compares recursive merge-sort implementations and benchmark results.
Examples
Uses a coffee shop, recursive merge sort with a sequential threshold, and parallel Go tests.
What you can do after
Evaluate parallel implementations against sequential baselines using benchmarks and profiling, and configure parallel Go tests.

Read how 100 Go Mistakes and How to Avoid Them explains parallelism →

Python Concurrency with asyncio

How it is used
Frames parallelization as a performance technique for distinguishing simultaneous execution from task switching in concurrent applications.
How deep it goes
Explains cores, the GIL, process pools, chunked MapReduce, reduction, shared state, and locks without detailing process internals.
Examples
Uses a two-baker cake analogy, multithreaded HTTP status reads, and parallel word-frequency MapReduce.
What you can do after
Build it from scratch by partitioning CPU-bound data, running chunks in process pools, gathering results, and merging partial outputs.

Read how Python Concurrency with asyncio explains parallelism →

Learn Concurrent Programming with Go

Parallelism is the simultaneous execution of multiple computations, typically on multiple processing resources.

How it is used
Distinguishes actual simultaneous execution from concurrent task planning and time-shared interleaving.
How deep it goes
Explains hardware requirements, scheduler interleaving, Go runtime handling, and Gustafson’s scaling perspective without runtime internals.
Examples
Uses request handling, file search, matrix rows, video rendering, I/O waiting, and multicore game workloads.
What you can do after
Evaluate/understand it by identifying whether tasks run simultaneously or are merely interleaved on available processing units.

Read how Learn Concurrent Programming with Go explains parallelism →

Kafka for Architects

Parallel processing is the simultaneous execution of multiple data-processing tasks to increase processing capacity or reduce elapsed time.

How it is used
Uses topic partitions and consumer groups to divide event-processing work across concurrent consumers.
How deep it goes
Explains partition-assignment limits, ordering, lag, worker-pool commits, and Kafka Streams tasks without a low-level implementation walkthrough.
Examples
Illustrates partition assignment with an address-change service and distributed stream processing with an orders application.
What you can do after
Operate/configure Kafka consumers and Kafka Streams instances with partition limits, ordering, and processing backlog in mind.

Read how Kafka for Architects explains parallelism →

Deep Learning with JAX

The execution of computation concurrently across multiple devices or cores.

How it is used
Deploys concurrent computation for large-scale training, simulations, and workloads containing independent operations.
How deep it goes
Explains pmap() parameters, SPMD execution, collectives, data parallelism, and multihost behavior, while deferring tensor-sharding details.
Examples
Demonstrates distributed dot-product calculations with sharded arrays and visualized tensor placement.
What you can do after
Apply pmap() to split arrays across devices, configure mapped axes and collectives, and implement data-parallel training.

Read how Deep Learning with JAX explains parallelism →

Apache Kafka in Action

Parallel processing is the simultaneous execution of work across multiple processing units or workers.

How it is used
Uses partitioned messages to increase processing throughput across servers and Kafka Streams instances.
How deep it goes
Explains partition-to-task assignment, instance distribution, threads, and failover; leaves Apache Flink’s different job management without detailed treatment.
Examples
Uses city-based trip data, product-price messages, a two-partition console demonstration, and a 12-partition filtering workload.
What you can do after
Operate/configure it by choosing independent partitioned work and distributing Kafka Streams tasks across instances and threads.

Read how Apache Kafka in Action explains parallelism →

Rust Servers, Services, and Apps

Parallelism is the simultaneous execution of multiple computations, typically on separate processors or cores.

How it is used
Uses parallel execution to improve processing speed and handle simultaneous requests efficiently across available processors.
How deep it goes
Defines simultaneous task execution and contrasts concurrency, illustrating CPU assignment without detailing parallelism implementation.
Examples
Illustrates two Rust functions assigned to separate CPUs and discusses processing multiple incoming requests.
What you can do after
Understand how simultaneous tasks can use multiple CPUs or cores to improve execution and request-processing efficiency.

Read how Rust Servers, Services, and Apps explains parallelism →

Timeless Algorithms, The Foundational Ideas

Parallel computation executes multiple computations concurrently across processing resources to reduce execution time or increase throughput.

How it is used
Enables Transformer training to process sequence tokens concurrently rather than propagate recurrent states one step at a time.
How deep it goes
Distinguishes concurrent training from sequential generation, then identifies system bottlenecks and diminishing returns without detailing implementation.
Examples
Contrasts parallel token processing during training with one-token-at-a-time generation and distributed computation across machines.
What you can do after
Evaluate why adding processors may not reduce elapsed time when memory, communication, coordination, or infrastructure constrain computation.

Read how Timeless Algorithms, The Foundational Ideas explains parallelism →

The Coder Cafe

Parallelization is the transformation of a workload so that independent portions can execute concurrently.

How it is used
Distinguishes simultaneous execution from independently organized work when deciding how to improve throughput.
How deep it goes
Defines parallelism through an analogy and use cases, then qualifies expected scaling without giving an implementation procedure.
Examples
Uses two waiters with separate coffee machines, parallel user requests, and an eight-thread CPU scaling estimate.
What you can do after
Evaluate/understand when simultaneous execution may improve throughput and when overhead or bottlenecks call for benchmarking.

Read how The Coder Cafe explains parallelism →

Functional Programming in C#, Second Edition

Parallel execution is the execution of multiple parts of a computation at the same time, typically on separate processing resources.

How it is used
Uses parallel processing to motivate pure functions and avoidance of shared state mutation.
How deep it goes
Works through PLINQ execution, a lost-update hazard, and a pure refactoring, while noting that removing state updates is not always straightforward.
Examples
Compares sequential and parallel list formatting, including sentence casing, a shared counter, and numbering with Zip.
What you can do after
Apply it via PLINQ to independent list operations and recognize shared counter updates that need protection or refactoring.

Read how Functional Programming in C#, Second Edition explains parallelism →

Data Pipelines with Apache Airflow, Second Edition

Execution of independent tasks concurrently rather than one after another.

How it is used
Runs independent data-pipeline branches together while respecting task dependencies and operational limits.
How deep it goes
Explains graph independence, asynchronous waiting, and specific concurrency controls, but does not provide a complete tuning procedure.
Examples
Uses independent weather and sales branches, two busy DAGs sharing a global limit, and external jobs monitored asynchronously.
What you can do after
Operate/configure it by identifying limiting Airflow settings and reserving worker capacity for tasks that need it.

Read how Data Pipelines with Apache Airflow, Second Edition explains parallelism →

Software Engineering for Data Scientists

Parallelization is the execution of computational work concurrently across multiple processing resources.

How it is used
Speeds data science workloads such as downloads, feature engineering, model training, prediction, and hyperparameter tuning.
How deep it goes
Defines parallel work, distinguishes vectorization and multiprocessing, and demonstrates n_jobs-based multi-core model training and tuning.
Examples
Contrasts 1,500 sequential and simultaneous downloads, then times random-forest training and hyperparameter searches on the artists dataset.
What you can do after
Operate/configure parallel model training and hyperparameter searches by choosing an appropriate n_jobs value for available CPUs.

Read how Software Engineering for Data Scientists explains parallelism →

Financial AI in Practice

Parallel processing is the simultaneous execution of multiple computational tasks across processing resources.

How it is used
Uses parallelism to prepare fraud graphs, analyze communities, and scale financial data pipelines.
How deep it goes
Shows executor-based workflows and conceptual pipeline designs, while leaving production-scale pipeline operation beyond scope.
Examples
Illustrates transaction edge preparation, community-level summaries, and concurrent clickstream and ledger extraction.
What you can do after
Apply it via Python's ThreadPoolExecutor to prepare transaction edges and analyze detected communities.

Read how Financial AI in Practice explains parallelism →

Learn Go with Pocket-Sized Projects

Parallel programming is a programming technique in which multiple computations execute concurrently to perform work at the same time.

How it is used
Uses parallelism to overlap independent work, especially branching exploration and background tasks.
How deep it goes
Explains goroutines, scheduling, communication, and tradeoffs, but defers detailed differences between goroutines and system threads.
Examples
Illustrates parallelism with cooking two pots and exploring alternative maze paths.
What you can do after
Apply it via Go goroutines and channels while judging independence, communication overhead, and uncertain performance gains.

Read how Learn Go with Pocket-Sized Projects explains parallelism →

The Cloud at Your Service: The when, how, and why of enterprise cloud computing

The execution of multiple tasks at the same time rather than sequentially.

How it is used
Connects parallelism to faster cloud testing and production performance, including ways to augment limited internal resources.
How deep it goes
Explains resource and workflow trade-offs at an overview level, with a functional-testing scenario rather than detailed implementation instructions.
Examples
Illustrates a 20-browser test farm and database shards distributed across physical instances.
What you can do after
Evaluate where concurrent testing resources may shorten feedback loops and where distributed data can support parallel work.

Read how The Cloud at Your Service: The when, how, and why of enterprise cloud computing explains parallelism →

Code Like a Pro in Rust

Parallelization is the execution of independent computational work concurrently across multiple threads or processing units.

How it is used
Uses parallelism to structure async tasks and optimize compute-intensive workloads.
How deep it goes
Demonstrates task scheduling and parallel iterators, but defers a detailed discussion of operating-system context switching.
Examples
Compares blocking and nonblocking sleeps, sum-of-squares reduction, and regular-expression searches.
What you can do after
Apply it via Tokio and Rayon, then benchmark sequential and parallel approaches for the workload.

Read how Code Like a Pro in Rust explains parallelism →

C# Concurrency: Asynchronous and multithreaded programming

Parallel execution is the simultaneous execution of multiple operations by separate processing resources.

How it is used
Uses parallel execution to finish independent processing sooner, especially when handling many client requests or item-level operations.
How deep it goes
Compares Thread, thread-pool, asynchronous, and Parallel implementations with benchmarks, but defers detailed queue design and delivery semantics.
Examples
Demonstrates mail merge, thread-per-message execution, sleeping-thread benchmarks, and asynchronous delay benchmarks.
What you can do after
Apply it via C# threads, thread-pool tasks, asynchronous APIs, or Parallel, then measure overhead and limit concurrency appropriately.

Read how C# Concurrency: Asynchronous and multithreaded programming explains parallelism →

Machines that Think

Parallel computation is the simultaneous execution of multiple computations rather than processing them strictly one after another.

How it is used
Explains how simultaneous computation supports large-scale language model training.
How deep it goes
Explains sequence-level and distributed parallelism and their constraints, without detailing specific parallelization algorithms.
Examples
Contrasts sequential LSTMs and GRUs with Transformers, and cites BERT as an example of training at larger scale.
What you can do after
Evaluate how architecture, hardware utilization, memory, and coordination affect the practicality of large-scale training.

Read how Machines that Think explains parallelism →

Building LLM Applications with DSPy

The overlapping execution of multiple operations so they can progress at the same time.

How it is used
Speeds language-model programs by overlapping module calls and evaluation requests.
How deep it goes
Shows asyncio calls, custom asynchronous modules, and concurrent subtasks; also discusses rate limits and debugging trade-offs.
Examples
Uses concurrent capital-city questions, a timed custom module, and an accounting-firm planning task split into subtasks.
What you can do after
Apply it via DSPy and asyncio to run module calls concurrently or configure threaded evaluation.

Read how Building LLM Applications with DSPy explains parallelism →

Bayesian Optimization in Action

Parallelism is the execution of independent processes simultaneously to reduce total completion time.

How it is used
Uses simultaneous evaluations to increase throughput when black-box objective queries are expensive.
How deep it goes
Defines independent concurrent execution and explains batch-query clustering, but stops before detailing batch BayesOpt policies.
Examples
Uses neural-network training, drug-recipe experiments, three-process timing, and clustered versus spread-out BayesOpt queries.
What you can do after
Apply it via batch BayesOpt by evaluating multiple selected query points simultaneously while avoiding redundant clustering.

Read how Bayesian Optimization in Action explains parallelism →

Learn Haskell by Example

Parallelism is the simultaneous execution of independent computations to reduce elapsed computation time.

How it is used
Improves multicore image processing by evaluating independent pixel transformations simultaneously.
How deep it goes
Explains independence, measurement, sparks, strategies, and chunking, but does not detail a complete parallel implementation.
Examples
Demonstrates grayscale, pixelation, and box-blur mappings applied across PNM images before PNG output.
What you can do after
Apply parallel evaluation strategies to uniform pure mappings and diagnose ineffective spark granularity.

Read how Learn Haskell by Example explains parallelism →

Grokking Simplicity

Execution in which multiple actions may occur simultaneously or without a fixed order relative to one another.

How it is used
Uses timeline diagrams to reason about ordering, timing, and correctness when asynchronous work runs across multiple timelines.
How deep it goes
Explains atomic decomposition, timeline construction, timing comparisons, and coordination, while leaving language-specific constraints to the applicable execution semantics.
Examples
Uses cost_ajax() and shipping_ajax() requests, response-time comparisons, and diagrams showing sequential and parallel timelines.
What you can do after
Diagnose ordering and timing bugs by decomposing actions, drawing timelines, and coordinating parallel work.

Read how Grokking Simplicity explains parallelism →

Event Processing in Action

The execution of separate parts of a computation simultaneously across multiple processing units.

How it is used
Improves performance by assigning independent event-processing activities to concurrent execution.
How deep it goes
Distinguishes three execution levels and identifies assignment trade-offs, but leaves complex decisions to system optimizers.
Examples
Uses stratified networks of independent agents and event-stream analysis distributed across computing nodes.
What you can do after
Understand concurrency levels and the factors an optimizer considers when grouping activities.

Read how Event Processing in Action explains parallelism →

Rust in Action: Systems programming concepts and techniques

Parallelism is the simultaneous execution of computational activities on multiple physical processing cores or other execution units.

How it is used
Distinguishes parallel execution from concurrency while introducing processes, threads, and related task abstractions.
How deep it goes
Defines parallelism and contrasts it with single-core concurrency, without explaining how to implement a parallel runtime.
Examples
Uses blocked threads waiting for memory or I/O to illustrate concurrency on a single CPU core.
What you can do after
Evaluate/understand the distinction between concurrent tasks and threads executing simultaneously on multiple cores.

Read how Rust in Action: Systems programming concepts and techniques explains parallelism →

Refactoring to Rust

Parallelism is the execution of multiple computations simultaneously to perform work concurrently.

How it is used
Uses parallelism to divide work across processor cores and improve application throughput.
How deep it goes
Defines parallelism against concurrency, illustrates throughput and ordering, and stops short of detailed synchronization mechanics.
Examples
Uses four threads on four cores, a two-hammer can crusher, and Python Mandelbrot tasks submitted through ThreadPoolExecutor.
What you can do after
Apply parallelism via ThreadPoolExecutor and asyncio.gather to distribute independent computations across execution threads.

Read how Refactoring to Rust explains parallelism →

Learn Docker in a Month of Lunches, Second Edition

Parallelism is the concurrent execution of multiple computational operations or data transfers.

How it is used
Optimizes Docker image pull times by distributing large data across concurrently downloaded layers.
How deep it goes
Explains concurrent layer downloads, configurable concurrency, Dockerfile alternatives, and transfer trade-offs.
Examples
Compares ML dataset images that copy days 70–79 into one layer or separate layers.
What you can do after
Evaluate Docker image transfer performance by building alternative targets and comparing sizes and layer histories.

Read how Learn Docker in a Month of Lunches, Second Edition explains parallelism →

Build an Advanced RAG Application (From Scratch)

Parallel processing is the simultaneous execution of computations on multiple parts of data or a sequence.

How it is used
Explains why Transformers process sequences efficiently and capture context more effectively than recurrent models.
How deep it goes
Defines parallel computation through Transformer self-attention and analogies, but stops short of implementation details or mathematical derivation.
Examples
Uses sentence processing, book reading, paragraph comprehension, and comparisons between recurrent and attention-based models.
What you can do after
Evaluate why Transformer models train and infer faster while capturing relationships across distant tokens.

Read how Build an Advanced RAG Application (From Scratch) explains parallelism →

Fast by Default

Parallelism is the simultaneous execution of multiple independent units of computation or work.

How it is used
Uses parallelism to shorten critical paths in service aggregation, rendering, initialization, and other latency-sensitive flows.
How deep it goes
Explains dependency mapping, deferred work, and asynchronous trade-offs, but stops short of language-specific implementation details.
Examples
Uses service aggregation, carrier-rate checks, independent page requests, and mobile startup data as concrete examples.
What you can do after
Diagnose avoidable serialization with traces, parallelize independent work, and compare its effect on the user-visible critical path.

Read how Fast by Default explains parallelism →

Microservices Patterns, Second Edition

Parallel execution runs independent saga steps concurrently rather than waiting for each step to finish before starting the next.

How it is used
Frames parallel execution as a saga-coordination choice alongside sequential execution.
How deep it goes
Defines concurrent execution of independent steps but does not detail implementation mechanics or selection criteria.

Read how Microservices Patterns, Second Edition explains parallelism →

Go in Action, Second Edition

Parallelism is the simultaneous execution of multiple computational activities, typically on separate processors or cores.

How it is used
Applies simultaneous execution to independent work, blocking I/O, CPU-bound computation, responsive applications, and real-time processing.
How deep it goes
Defines parallelism against concurrency and demonstrates goroutine-based execution, but defers detailed synchronization and completion coordination.
Examples
Uses an RSS reader that fetches New York Times and BBC feeds through separate goroutines.
What you can do after
Apply it via Go goroutines, channels, and worker pools to overlap independent work and use multiple CPU cores.

Read how Go in Action, Second Edition explains parallelism →

The Complete Obsolete Guide to Generative AI

Parallel computation is the simultaneous execution of multiple computational operations.

How it is used
Accelerates generative AI and other computationally intensive workloads by executing compatible operations concurrently.
How deep it goes
Defines parallel computation and connects it to AI acceleration, but stops short of explaining scheduling or implementation details.
Examples
Uses AI matrix calculations, neural-network operations, specialized accelerators, and qubits in superposition as examples.
What you can do after
Evaluate how parallel processing and specialized hardware can improve AI training, inference speed, energy use, and deployment feasibility.

Read how The Complete Obsolete Guide to Generative AI explains parallelism →

GPU Programming with Triton

Parallelism is the simultaneous execution of multiple tasks by multiple physical or computational resources.

How it is used
Distinguishes simultaneous execution from overlapping task progress when explaining how GPU workloads use multiple resources.
How deep it goes
Defines parallelism, contrasts it with concurrency, and illustrates the distinction without detailing kernel implementation.
Examples
Uses a coffee shop with one or several cashiers and GPU threads processing independent work.
What you can do after
Understand how independent tasks can be structured for simultaneous execution on multiple resources.

Read how GPU Programming with Triton explains parallelism →

AI Agents in Action, Second Edition

The concurrent execution of independent workloads or subtasks to reduce elapsed processing time.

How it is used
Speeds independent workloads in multi-agent systems and repetitive task loops.
How deep it goes
Explains fan-out execution, asyncio.gather, batching, timing comparisons, rate limits, and independence checks.
Examples
Uses company research, pull-request review, document batches, and five text-transformation tasks as examples.
What you can do after
Apply it via asyncio.gather to compare sequential and concurrent task processing while monitoring failures and rate limits.

Read how AI Agents in Action, Second Edition explains parallelism →

Deep Learning with PyTorch, Second Edition

The practice of dividing work into tasks that execute simultaneously.

How it is used
Accelerates machine-learning training by dividing data or computation across simultaneous tasks, processes, devices, or threads.
How deep it goes
Explains task decomposition, thread-process differences, collective communication, and engineering challenges without detailing every parallelism strategy.
Examples
Uses a ten-call timing comparison, a jigsaw analogy, PyTorch broadcast and all-reduce, and TorchTitan configurations.
What you can do after
Apply it via concurrent.futures and PyTorch distributed operations to compare execution modes and coordinate tensor data across processes.

Read how Deep Learning with PyTorch, Second Edition explains parallelism →

Effective Data Analysis

Parallel processing is the simultaneous execution of separate portions of a computational task.

How it is used
Accelerates analytics, statistical modeling, and machine learning on datasets too large or slow for local processing.
How deep it goes
Explains task division, cluster execution, result recombination, and fault tolerance, but stops short of implementation details.
Examples
Uses Spark SQL, PySpark, and a task-partitioning figure as concrete examples of parallel data processing.
What you can do after
Apply it via distributed computing frameworks while accounting for framework syntax, resource limits, and processing costs.

Read how Effective Data Analysis explains parallelism →

Learn Generative AI with PyTorch

A training approach that computes multiple sequence elements simultaneously rather than strictly sequentially.

How it is used
Accelerates Transformer training so models can learn from large datasets and support large language models.
How deep it goes
Explains simultaneous versus sequential processing and positional encodings, but stops short of device-level implementation details.
Examples
Uses Transformer language translation and contrasts it with sequential RNN processing.
What you can do after
Evaluate why Transformers train faster than RNNs and why positional encodings accompany parallel input processing.

Read how Learn Generative AI with PyTorch explains parallelism →

Spring AI in Action

Parallelization is a technique that executes multiple processing paths concurrently and aggregates their results.

How it is used
Uses parallel paths to answer related questions concurrently within an agent workflow before producing an aggregate response.
How deep it goes
Implements worker execution, thread-pool sizing, result aggregation, summarization, and Spring workflow wiring.
Examples
Demonstrates Azul analysis by running player-count and game-mechanics actions before summarizing their outputs.
What you can do after
Apply it via Java CompletableFuture, executor services, injected actions, and Spring bean configuration.

Read how Spring AI in Action explains parallelism →

C++ Concurrency in Action, Second Edition

Parallelism is the simultaneous execution of multiple tasks, typically to use available hardware and improve performance.

How it is used
Uses simultaneous tasks to exploit available hardware and improve bulk data-processing performance.
How deep it goes
Defines the performance-oriented distinction from concurrency and points to library-managed parallelization without detailing its mechanics.
Examples
Uses networked processes and standard-library algorithm calls to illustrate ways to increase or delegate parallel work.
What you can do after
Evaluate parallelism as a performance goal and identify standard-library algorithms as a route to delegating task division.

Read how C++ Concurrency in Action, Second Edition explains parallelism →

Introduction to Generative AI, Second Edition

Parallelization is the execution of independent subtasks or tool calls concurrently rather than sequentially.

How it is used
Uses concurrent actions to reduce the slowness of sequential tool execution in complex agent tasks.
How deep it goes
Explains the principle and task dependencies through a workflow example, but does not detail implementation mechanics.
Examples
Uses a pickleball news website whose frontend work and API research proceed in parallel.
What you can do after
Apply it by separating independent subtasks from actions whose inputs depend on earlier results.

Read how Introduction to Generative AI, Second Edition explains parallelism →

Outlier Detection in Python

Parallelism is the execution of multiple computational operations concurrently rather than one after another.

How it is used
Reduces outlier-detection training or prediction time by executing independent work concurrently.
How deep it goes
Explains process controls, detector independence, batch splitting, and sequential trade-offs without detailing parallel-runtime internals.
Examples
Uses KDD Cup data, concurrent batch prediction, and sequential Isolation Forest, LOF, and GMM filtering.
What you can do after
Apply it via n_jobs, multiprocessing, SUOD, or concurrent batch execution while comparing sequential workload reduction.

Read how Outlier Detection in Python explains parallelism →

AI Agents and Applications

Parallel processing is the simultaneous execution of the same or related operations on multiple independent inputs.

How it is used
Accelerates document summarization and web research by executing independent chain operations concurrently.
How deep it goes
Explains chunk and search parallelization through map() and composition, while stopping short of low-level scheduling internals.
Examples
Uses document chunks, parallel web searches, an Astorga research query, and LCEL chain diagrams and listings.
What you can do after
Apply it via LangChain and LCEL to parallelize independent summarization or research steps and aggregate their outputs.

Read how AI Agents and Applications explains parallelism →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Parallelization is the execution of independent computational operations concurrently across multiple processing resources.

How it is used
Accelerates XGBoost training by distributing each boosting model’s feature-column computations across multiple CPU cores.
How deep it goes
Explains column blocks and serial boosting, but stops short of detailing scheduling or split-finding algorithms.
Examples
Uses XGBoost CPU utilization and column-block training as concrete examples of parallel execution.
What you can do after
Operate XGBoost through its Scikit-learn API while benefiting from internal multi-core training.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains parallelism →

Claude at Work

Parallelism is the execution of multiple independent development tasks or plans at the same time.

How it is used
Accelerates software delivery by delegating independent work or overlapping planning and review with implementation.
How deep it goes
Distinguishes subagent delegation, separate Claude Code instances, and parallel planning while emphasizing dependency limits.
Examples
Uses per-component test delegation and reviewing one feature slice's plan while another is built.
What you can do after
Apply via Claude Code by assigning independent tasks to subagents or coordinating work across separate sessions.

Read how Claude at Work explains parallelism →

Functional Design and Architecture: Examples in Haskell

The simultaneous execution of multiple computational activities.

How it is used
Frames parallelism as a design goal that influences early choices of tools and approaches.
How deep it goes
Explains why functional techniques ease reasoning about parallelism, but stops before implementation details.
What you can do after
Evaluate programming approaches for how simply they support reasoning about parallelism and correctness.

Read how Functional Design and Architecture: Examples in Haskell explains parallelism →

High Performance Unity Game Development

Executing computational work concurrently by dividing it into tasks.

How it is used
Uses multiple CPU cores by dividing a workload into tasks to improve performance.
How deep it goes
Defines task splitting across cores and notes scheduling overhead, without detailing job implementation.
Examples
Illustrates parallel updates of enemy GameObject positions with Jobs and TransformAccessArray.
What you can do after
Apply it via Unity Jobs and TransformAccessArray to parallelize GameObject transform updates when they are the bottleneck.

Read how High Performance Unity Game Development explains parallelism →

Also covered in

What to know first

Related concepts

Questions about parallelism

What is parallelism?

Parallelism is the execution of multiple independent tasks or computational operations at the same time rather than sequentially.

Which Manning books explain parallelism?

Fast Python: High performance techniques for large datasets, Grokking Concurrency, CUDA for LLMs, Latency, Functional Programming in Scala, Second Edition, Starting Data Analytics with Generative AI and Python, Advanced Algorithms and Data Structures, Graph Neural Networks in Action and 47 more.

What should I know before learning parallelism?

Manning books that teach parallelism usually introduce thread, language model, CPU and Python first.

What is the difference between parallelism and concurrency?

Parallelism is the execution of multiple independent tasks or computational operations at the same time rather than sequentially. Concurrency is the structuring or execution of multiple computational activities so their progress overlaps in time, whether or not they run simultaneously.

What kinds of parallelism are there?

Kinds of parallelism include file-level parallelism.

What is parallelism also called?

Parallelism is also called parallelization, parallel execution, parallel processing, parallel computing or parallel computation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.