7 Handling blockingwork with threads

 

This chapter covers

  • Reviewing the multithreading library
  • Creating thread pools to handle blocking I/O
  • Using async and await to manage threads
  • Handling blocking I/O libraries with thread pools
  • Handling shared data and locking with threads
  • Handling CPU-bound work in threads

When developing a new I/O-bound application from scratch, asyncio may be a natural technology choice. From the beginning, you’ll be able to use non-blocking libraries that work with asyncio, such as asyncpg and aiohttp, as you begin development. However, greenfields (a project lacking constraints imposed by prior work) development is a luxury that many software developers don’t have. A large portion of our work may be managing existing code using blocking I/O libraries, such as requests for HTTP requests, psycopg for Postgres databases, or any number of blocking libraries. We may also be in a situation where an asyncio-friendly library does not yet exist. Is there a way to get the performance gains of concurrency while still using asyncio APIs in these cases?

Multithreading is the solution to this question. Since blocking I/O releases the global interpreter lock, this enables the possibility to run I/O concurrently in separate threads. Much like the multiprocessing library, asyncio exposes a way for us to utilize pools of threads, so we can get the benefits of threading while still using the asyncio APIs, such as gather and wait.

7.1 Introducing the threading module

 

7.2 Using threads with asyncio

 
 

7.2.1 Introducing the requests library

 
 

7.2.2 Introducing thread pool executors

 

7.2.3 Thread pool executors with asyncio

 
 

7.2.4 Default executors

 
 
 

7.3 Locks, shared data, and deadlocks

 
 
 

7.3.1 Reentrant locks

 
 
 

7.3.2 Deadlocks

 
 
 

7.4 Event loops in separate threads

 
 

7.4.1 Introducing Tkinter

 
 
 
 

7.4.2 Building a responsive UI with asyncio and threads

 
 
 

7.5 Using threads for CPU-bound work

 
 
 

7.5.1 Multithreading with hashlib

 
 
 

7.5.2 Multithreading with NumPy

 

Summary

 
 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest