Multiprocessing
Multiprocessing is a programming technique that executes work in multiple operating-system processes, enabling concurrent or parallel execution across available processor resources.
Python Concurrency with asyncio
Multiprocessing is a concurrency technique in which multiple processes perform work concurrently, typically with a parent process managing child processes.
- How it is used
- Uses multiprocessing to parallelize CPU-bound work and extend asyncio when asynchronous tasks would block the event loop.
- How deep it goes
- Explains parent-child processes, start and join, process pools, serialization, shared state, and event-loop-per-process designs.
- Examples
- Demonstrates child-process greetings, Fibonacci computation, pooled counting, and database queries handled by one event loop per process.
- What you can do after
- Builds process-based workers, distribute CPU-bound calls through ProcessPoolExecutor, and combine process pools with asyncio.
Read how Python Concurrency with asyncio explains multiprocessing →
Fast Python: High performance techniques for large datasets
- How it is used
- Uses multiprocessing to parallelize pure Python MapReduce work across CPU cores despite the GIL.
- How deep it goes
- Explains executor substitution, serialization failures, communication overhead, and granularity, while deferring lower-level language solutions.
- Examples
- Demonstrates a MapReduce process pool whose lambda counter raises a PicklingError, then replaces it with a def function.
- What you can do after
- Apply it via concurrent.futures.ProcessPoolExecutor and diagnose serialization failures using future exceptions.
Read how Fast Python: High performance techniques for large datasets explains multiprocessing →
Software Engineering for Data Scientists
Multiprocessing is the execution of computational tasks in separate operating-system processes, typically enabling use of multiple CPU cores.
- How it is used
- Speeds independent data-processing or collection tasks by running Python work in parallel.
- How deep it goes
- Demonstrates CPU checking, Pool creation, map execution, and shutdown, without detailing operating-system process internals.
- Examples
- Uses pytesseract OCR across receipt-image files, comparing sequential processing with a three-process Pool.
- What you can do after
- Apply it via Python’s multiprocessing library to map an independent function across files with a bounded process pool.
Read how Software Engineering for Data Scientists explains multiprocessing →
Learn Concurrent Programming with Go
An operating-system capability in which multiple tasks are managed so that more than one can make progress on available processor resources.
- How it is used
- Keeps processor resources productive by switching to other tasks when the current task waits for input or I/O.
- How deep it goes
- Explains job, ready, running, and I/O waiting states, then demonstrates process coordination through a buffered pipe.
- Examples
- Uses a large-file string search and a curl-to-wc UNIX pipeline to illustrate scheduling, process IDs, buffering, and termination.
- What you can do after
- Evaluate/understand how operating systems schedule tasks and coordinate command-line processes through pipes.
Read how Learn Concurrent Programming with Go explains multiprocessing →
Also covered in
What to know first
- PythonPython is a high-level, general-purpose, dynamically typed programming language with readable syntax and support for multiple programming paradigms.
- threadA thread is a distinct sequence of execution with its own control flow and state, managed or scheduled by a processor, runtime, or operating system.
- workflowA workflow is an organized sequence of coordinated activities or tasks through which work progresses toward a defined outcome.
- CPUA central processing unit (CPU) is a computer processor that executes program instructions and performs general-purpose computational operations.
Related concepts
- global interpreter lockcontrasts with · The global interpreter lock is a mutual-exclusion mechanism in CPython that permits only one thread at a time to execute Python bytecode within a process.
- parallelismkind of · Parallelism is the execution of multiple independent tasks or computational operations at the same time rather than sequentially.
- compute-bounduses · A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
- concurrent.futurescontrasts with · concurrent.futures is a Python standard-library module providing high-level interfaces for asynchronously executing and coordinating callable tasks with threads or processes.
- process poolhas part · A process pool is a managed collection of worker processes that executes submitted tasks concurrently.
- operating systemused by · An operating system is system software that manages computer hardware resources and provides services and interfaces for application programs.
Questions about multiprocessing
What is multiprocessing?
Multiprocessing is a programming technique that executes work in multiple operating-system processes, enabling concurrent or parallel execution across available processor resources.
Which Manning books explain multiprocessing?
Python Concurrency with asyncio, Fast Python: High performance techniques for large datasets, Software Engineering for Data Scientists and Learn Concurrent Programming with Go.
What should I know before learning multiprocessing?
Manning books that teach multiprocessing usually introduce Python, thread, workflow and CPU first.
What is the difference between multiprocessing and global interpreter lock?
Multiprocessing is a programming technique that executes work in multiple operating-system processes, enabling concurrent or parallel execution across available processor resources. The global interpreter lock is a mutual-exclusion mechanism in CPython that permits only one thread at a time to execute Python bytecode within a process.
What is multiprocessing a kind of?
Multiprocessing is a kind of parallelism. Parallelism is the execution of multiple independent tasks or computational operations at the same time rather than sequentially.
What are the parts of multiprocessing?
Multiprocessing includes process pool.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



