liveBook · Manning

Process.communicate

As explained in Python Concurrency with asyncio

Process.communicate is a coroutine method that waits for a subprocess to finish while concurrently consuming its standard output and standard error.

Process.communicate is an asyncio coroutine that waits for a subprocess to finish while concurrently consuming standard output and standard error. By draining both streams as the process runs, it prevents the pipe-buffer deadlocks that can occur when a subprocess produces substantial output. section 13.1.1

The coroutine returns the collected standard output and standard error after completion. When standard input is configured as a pipe, passing input bytes to communicate sends them to the subprocess, making it suitable for a single known input exchange such as encrypting text or answering one username prompt. section 13.1.2 section 13.2

Choose communicate when:

  • You need complete output after a subprocess finishes and do not need to react to output while it runs. section 13.1.1
  • You want to send one known byte sequence through standard input, such as plaintext passed to gpg for encryption. section 13.1.2
  • You need to run multiple independent subprocess operations concurrently; combine communicate tasks with asyncio.gather, and use a semaphore when their number is large. section 13.1.2 section 13.3

Collecting output safely

After creating a subprocess with stdout and stderr pipes, await `stdout, stderr = await process.communicate()`. Print the returned values and read `process.returncode` after completion. This pattern safely handles a program that generates a large amount of output, but it retains the collected data in memory. section 13.1.1

How Python Concurrency with asyncio covers it

How deep it goes
Explains concurrent stream consumption and deadlock prevention, but defers interactive protocols and large-output streaming to other patterns.
Examples
Uses large-output Python programs, gpg encryption, a username prompt, and echo subprocesses to contrast finite and interactive communication.
What it assumes
Assumes basic asyncio subprocess creation and pipes, while establishing why stream consumption matters for avoiding deadlocks.
What you can do after
Apply it via asyncio to run subprocesses safely, send finite input, collect output, and avoid pipe-related deadlocks.
How it connects
Connects subprocesses with standard input, standard output, standard error, pipes, wait, asyncio.gather, and semaphores.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Python Concurrency with asyncio. The section links point to the parts of the book it draws on.