13 Managing subprocesses

 

This chapter covers

  • Running multiple subprocesses asynchronously
  • Handling standard output from subprocesses
  • Communicating with subprocesses using standard input
  • Avoiding deadlocks and other pitfalls with subprocesses

Many applications will never need to leave the world of Python. We’ll call code from other Python libraries and modules or use multiprocessing or multithreading to run Python code concurrently. However, not everything we’ll want to interact with is written in Python. We may have an already built application that is written in C++, Go, Rust, or some other language that provides better runtime characteristics or is simply already there for us to use without reimplementing. We may also want to use OS provided command-line utilities, such as GREP for searching through large files, cURL for making HTTP requests, or any of the numbers of applications we have at our disposal.

In standard Python, we can use the subprocess module to run different applications in separate processes. Like most other Python modules, the standard subprocess API is blocking, making it incompatible with asyncio without multithreading or multiprocessing. asyncio provides a module modeled on the subprocess module to create and manage subprocesses asynchronously with coroutines.

13.1 Creating a subprocess

13.1.1 Controlling standard output

13.1.2 Running subprocesses concurrently

13.2 Communicating with subprocesses

Summary