Imagine sending a rubber duck down a pipe: one task puts it in, and another receives it. Message passing works similarly. Tasks communicate by sending and receiving data rather than directly sharing memory. It is a form of interprocess communication (IPC), even when the communicating tasks are threads. section 5.1 section 5.3
In an operating-system-managed channel, a task sends a message to a named task, and the receiver reads it. The OS provides the channel and send-and-receive interfaces, reducing conflicts over the channel. That convenience costs time: data is copied from the sender into the OS channel and then into the receiver’s address space. Message passing is therefore not an automatic performance win. section 5.3
Common ways to pass messages include:
- Pipes: data flows from a writer to a reader in one direction; two pipes are needed for two-way communication. section 5.3
- Message queues: workers take items from a queue to process them. section 5.3
- Sockets: a sender and receiver exchange data through a connection. section 5.3
A duck through a Python pipe
A writer calls send("Rubber duck") on one Pipe connection; a reader calls recv() on the other. The example makes the handoff visible without requiring either task to read the other’s variables. section 5.3
# Chapter 5/pipe.py
from threading import Thread, current_thread
from multiprocessing import Pipe
from multiprocessing.connection import Connection
class Writer(Thread):
def __init__(self, conn: Connection):
super().__init__()
self.conn = conn
self.name = "Writer"
def run(self) -> None:
print(f"{current_thread().name}: Sending rubber duck...")
self.conn.send("Rubber duck") #1
class Reader(Thread):
def __init__(self, conn: Connection):
super().__init__()
self.conn = conn
self.name = "Reader"
def run(self) -> None:
print(f"{current_thread().name}: Reading...")
msg = self.conn.recv() #2
print(f"{current_thread().name}: Received: {msg}")
def main() -> None:
reader_conn, writer_conn = Pipe() #3
reader = Reader(reader_conn)
writer = Writer(writer_conn)
threads = [
writer,
reader
]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
if __name__ == "__main__":
main()
Inspect the writer’s send() and reader’s recv() calls to see the message cross the pipe. section 5.3
Keeping separate copies of data can help avoid locks around shared memory. The trade-off is communication cost, so choose the approach to fit the work. Message passing can also extend beyond one machine, connecting it to distributed systems and client-server applications. section 9.10 section 5.3 section 10.0
