How deeply does Python Concurrency with asyncio cover run_in_executor?

Explains callable submission, awaitable results, partial arguments, worker queues, executor defaults, and coordination with gather.

Run_in_executor

As explained in Python Concurrency with asyncio

run_in_executor is an asyncio event-loop method that schedules a synchronous callable to run in an executor and returns an awaitable for its result.

run_in_executor is an asyncio event-loop method for running a synchronous callable in a thread or process executor. It returns an awaitable, so the result can be awaited directly or coordinated with asyncio.gather. section 6.3.2 section 7.2.3

The method accepts the executor and a zero-argument callable. Use functools.partial to freeze arguments before submission; this adapts functions such as count(number) or get_status_code(url). section 6.3.2 section 7.2.3

Choose the executor according to the blocking work:

Use a process pool to parallelize CPU-bound operations, such as counting tasks or partitioned MapReduce work. section 6.3.2 section 6.4.3
Use a thread pool to move blocking operations, such as synchronous HTTP requests, off the event-loop thread while other asynchronous code runs. section 7.2.3
Pass None to reuse the event loop’s default ThreadPoolExecutor; set a custom default with loop.set_default_executor when needed. section 7.2.4

Run synchronous requests concurrently

async def main(): loop = asyncio.get_running_loop() urls = ['https://www.example.com' for _ in range(1000)] tasks = [loop.run_in_executor( None, functools.partial(get_status_code, url) ) for url in urls] results = await asyncio.gather(*tasks) print(results)

How Python Concurrency with asyncio covers it

Cover of Python Concurrency with asyncio

liveBook manning.com

Read Python Concurrency with asyncio on liveBook Get the book at manning.com

Related concepts

process pooluses · A process pool is a managed collection of worker processes that executes submitted tasks concurrently.
awaitableproduces · An awaitable is a Python object that can be used with the await expression to suspend a coroutine until the asynchronous operation completes and produce its result.
partial applicationaddressed by · Partial application is the creation of a new function by binding some arguments of an existing function while leaving the remaining arguments to be supplied later.
thread pooluses · A worker pool is a managed collection of workers that execute submitted or queued tasks, typically reusing workers to handle multiple tasks concurrently.

This explainer was generated by Manning's knowledge graph from the text of Python Concurrency with asyncio. The section links point to the parts of the book it draws on.