Shared memory
Shared memory is a memory region or storage area accessible to multiple processes, threads, or other execution components for accessing or exchanging data.
Fast Python: High performance techniques for large datasets
- How it is used
- Uses shared storage to avoid array copies or make objects available to separate local processes.
- How deep it goes
- Works through NumPy sharing checks and Plasma setup, but defers naive shared-memory and low-level-lock implementations.
- Examples
- Uses a flipped image view and copy, plus a CSV-to-Plasma producer and separate consumers.
- What you can do after
- Apply it via NumPy sharing checks and Arrow Plasma object storage while recognizing mutation and coordination trade-offs.
Read how Fast Python: High performance techniques for large datasets explains shared memory →
Grokking Concurrency
A shared variable is a variable whose value can be accessed by multiple concurrent tasks.
- How it is used
- Enables concurrent tasks to exchange data through common memory, particularly when copying large amounts of data is costly.
- How deep it goes
- Defines shared memory, illustrates it with Python threads, and weighs speed against access conflicts without detailing allocation internals.
- Examples
- Uses a shared refrigerator, a Python producer–consumer array, and matrix tasks reading common inputs.
- What you can do after
- Apply it via Python threading to exchange data through a shared array while recognizing the need to coordinate conflicting access.
Related concepts
- memory hierarchypart of · A memory hierarchy is an arrangement of storage levels with differing capacities, access speeds, latencies, costs, or persistence.
- message passingalternative to · A communication or computation technique in which entities exchange messages to transmit or propagate information, coordinate work, or update state.
- gpu video memorycontrasts with · GPU memory is memory accessible to a graphics processing unit for storing data and computation state used in graphics or other workloads.
- reduction operationused by · A reduction operation combines multiple input values or partial results according to a combining rule to produce fewer results, often a single aggregate.
- channelalternative to · A channel is a medium, mechanism, or pathway through which information or messages are conveyed between participants or endpoints.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.