asyncio.subprocess.PIPE is a sentinel passed to a subprocess stream parameter. For stdout, it creates an asyncio-managed StreamReader that the application can consume asynchronously. section 13.1.1
Use the redirected stream to:
- Read subprocess output line by line and add context, such as the command that produced each line. section 13.1.1
- Consume output concurrently with process.wait() when running commands that produce manageable streams. section 13.1.1
- Use communicate() when collecting piped output until the subprocess terminates. section 13.1.1 section 13.2
Pipes require active consumption. If a subprocess writes more data than the pipe can buffer and the parent only awaits wait(), both processes can hang. Consume the stream concurrently or use communicate(). section 13.1.1
Prefix output from ls
Pass stdout=asyncio.subprocess.PIPE to create_subprocess_exec, then give process.stdout to a task that awaits readline(), decodes each line, and prints it with an [ls -la] prefix. section 13.1.1
Listing 13.3 Using the standard output stream reader
import asyncio
from asyncio import StreamReader
from asyncio.subprocess import Process
async def write_output(prefix: str, stdout: StreamReader):
while line := await stdout.readline():
print(f'[{prefix}]: {line.rstrip().decode()}')
async def main():
program = ['ls', '-la']
process: Process = await asyncio.create_subprocess_exec(*program,
stdout=asyncio
.subprocess.PIPE)
print(f'Process pid is: {process.pid}')
stdout_task = asyncio.create_task(write_output(' '.join(program), process.stdout))
return_code, _ = await asyncio.gather(process.wait(), stdout_task)
print(f'Process returned: {return_code}')
asyncio.run(main())
Shows the complete pattern: create a subprocess with stdout=PIPE, consume process.stdout in a task, and await consumption alongside process.wait(). section 13.1.1
