Plasma server

Q: How deeply does Fast Python: High performance techniques for large datasets cover Plasma server?

Explains server startup, object identifiers, catalog discovery, storage, retrieval, and a three-process workflow, while treating the design as illustrative.

As explained in Fast Python: High performance techniques for large datasets

Plasma is an in-memory object store and server associated with Apache Arrow that enables processes to exchange objects efficiently.

Plasma is a local in-memory object store associated with Apache Arrow. It lets separate processes register, discover, read, and write shared objects efficiently, using Arrow’s common in-memory representation and zero-copy-oriented design. section 7.5 section 7.5.2

A typical Plasma workflow separates data loading, analysis, and result display:

Start a local server with a UNIX socket and a configured shared-memory capacity. section 7.5.2
Load a CSV file with Arrow, assign the resulting table a 20-byte ObjectID, and store it with the Plasma client. section 7.5.2
Let a worker discover objects through the catalog, retrieve the table, compute results, and publish a result object for another process to read. section 7.5.2

Shared NYC taxi analysis

A producer reads the NYC taxi CSV into an Arrow table and places it in Plasma. An analysis process locates the csv- object, computes statistics, and stores a result- object; a display process lists and retrieves the result. This separates expensive processing from analysis code and supports processes that cannot be linked directly into Python. section 7.5.2

How Fast Python: High performance techniques for large datasets covers it

Cover of Fast Python: High performance techniques for large datasets

liveBook manning.com

Read Fast Python: High performance techniques for large datasets on liveBook Get the book at manning.com

Related concepts

shared memoryuses · Shared memory is a memory region accessible by multiple execution contexts for accessing or exchanging data.
serverkind of · A server is a computer system or software process that receives requests from clients or other systems and provides resources, data, or services in response.
interprocess communicationuses · Interprocess communication is the exchange of data, information, or coordination signals between independently executing processes.
identifierconfigured by · An identifier is a name or value that denotes or distinguishes an entity or program element within a system.
Apache Arrowpart of · Apache Arrow is an open, language-independent specification and software ecosystem for representing and processing columnar data in memory.

This explainer was generated by Manning's knowledge graph from the text of Fast Python: High performance techniques for large datasets. The section links point to the parts of the book it draws on.