Multiprocessing runs multiple instances of an application at the same time to finish a task faster. In Python, that means launching separate Python processes and dividing work among them. It is a form of parallelization: instead of waiting for one operation to finish before starting the next, independent operations can proceed simultaneously. This can help when a data science workflow must perform many expensive operations, such as collecting data, processing a large file collection, or generating results under low-latency expectations. section 6.0 section 6.2 section 6.7
A process is an instance of a program, whereas a thread is a smaller unit within a process that shares that process’s resources. Multiprocessing therefore differs from multithreading, asynchronous programming, and vectorization. Choose it when the same function can be applied independently to many inputs and separate Python processes can handle those inputs in parallel. section 6.2 section 6.2.2
A practical multiprocessing workflow is:
- Check available CPUs with multiprocessing.cpu_count(). The number of concurrent processes is limited by available CPUs. section 6.2.2
- Put the repeated work in a function that accepts one input, such as a filename. section 6.2.2
- Create a multiprocessing.Pool with a chosen process count. When four cores are assumed, a pool of three leaves one core free to avoid consuming too many resources. section 6.2.2
- Use pool.map to apply the function across the input list, then close the pool to shut down the Python processes. section 6.2.2
OCR receipt images in parallel
Optical character recognition (OCR) provides a focused case for multiprocessing. A sequential workflow reads each receipt image with cv2 and calls pytesseract.image_to_string one file at a time; the illustrated run takes about 185 seconds. The parallel version keeps the same scrape_text function but creates Pool(3) and maps that function over the image files. Separate Python processes handle OCR work, and their results are combined. The example runs in roughly half the time, showing how parallelizing independent files can shorten a batch job. section 6.2.2
Figure 6.4 This diagram visualizes how Python’s multiprocessing package creates separate Python processes to tackle the OCR task, followed by combining the results from each individual process. The result of doing this for the OCR task is much faster than using just a single process.
This process diagram makes the division of OCR work visible: separate Python processes handle individual tasks before their results are combined. section 6.2.2
Faster execution is not automatic: multiprocessing is one option within a broader efficiency toolkit. Vectorization operates on collections at once, caching trades memory for saved computation, and profiling helps find slow code locations. Multiprocessing is specifically useful when work can be split into concurrent Python-process tasks, such as data collection or image-text extraction. section 6.2 section 6.7



