Imagine putting a rubber duck into a water pipe: it enters upstream and comes out downstream. A pipe moves data in much the same way. One task writes to an end, and another reads from the other end. The flow goes one way; two pipes are needed if both tasks must send data to each other. section 5.3
Pipes are a form of message-passing interprocess communication (IPC). They provide a simple, synchronized way for a producer to pass data to a consumer without both tasks accessing the same shared memory. section 5.3 section 5.6
There are two kinds of pipe:
- An unnamed pipe connects related tasks, such as parent and child processes, that share file descriptors. section 5.3
- A named pipe offers a simple interface for transferring data between two processes. section 5.6
Sending a rubber duck
A Python example creates two connections with multiprocessing.Pipe(). A writer thread calls send("Rubber duck") on one connection; a reader thread calls recv() on the other and prints the message. The example makes the two ends—and the direction of travel—visible. section 5.3
# Chapter 5/pipe.py
from threading import Thread, current_thread
from multiprocessing import Pipe
from multiprocessing.connection import Connection
class Writer(Thread):
def __init__(self, conn: Connection):
super().__init__()
self.conn = conn
self.name = "Writer"
def run(self) -> None:
print(f"{current_thread().name}: Sending rubber duck...")
self.conn.send("Rubber duck") #1
class Reader(Thread):
def __init__(self, conn: Connection):
super().__init__()
self.conn = conn
self.name = "Reader"
def run(self) -> None:
print(f"{current_thread().name}: Reading...")
msg = self.conn.recv() #2
print(f"{current_thread().name}: Received: {msg}")
def main() -> None:
reader_conn, writer_conn = Pipe() #3
reader = Reader(reader_conn)
writer = Writer(writer_conn)
threads = [
writer,
reader
]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
if __name__ == "__main__":
main()
Follow the writer’s send() call and the reader’s recv() call to see how a message crosses the pipe. section 5.3
Choose a pipe when two tasks need straightforward, synchronous producer–consumer communication. Other IPC mechanisms suit different needs: a message queue supports asynchronous exchange, while a socket provides a two-way channel. A pipe is temporary and closes if either its sender or receiver half is dropped. section 5.3 section 5.6
