asyncio.gather runs a sequence of awaitables concurrently and returns an awaitable that finishes only after every supplied operation completes. Awaiting it produces a list of results; coroutine arguments are automatically wrapped in tasks, so separate create_task calls are unnecessary. section 4.4
Use gather when a group of operations should run together and the next step needs the whole group.
- Build a list of coroutines, expand it into gather, and await the returned result. section 4.4
- Expect all operations to finish before processing the returned results; use as_completed instead when results should be handled as soon as they arrive. section 4.4 section 4.7
- Leave return_exceptions false to propagate the first observed exception through await, or set it true to receive exceptions alongside successful results. section 4.4.1
- Recognize that a failed awaitable does not automatically cancel the other running awaitables. section 4.4.1
- Use gather beyond HTTP requests, including executor tasks, subprocesses, and custom task runners. section 7.2.3 section 13.1.2 section 14.1
Fetching many status codes concurrently
Create one coroutine per URL, then await them as a group: requests = [fetch_status(session, url) for url in urls]; status_codes = await asyncio.gather(*requests). The same structure can launch 1,000 requests concurrently and collect their status codes after all requests finish. For mixed success and failure, add return_exceptions=True and separate Exception instances from successful values. section 4.4 section 4.4.1
Listing 4.6 Running requests concurrently with gather
import asyncio
import aiohttp
from aiohttp import ClientSession
from chapter_04 import fetch_status
from util import async_timed
@async_timed()
async def main():
async with aiohttp.ClientSession() as session:
urls = ['https:/ / example .com' for _ in range(1000)]
requests = [fetch_status(session, url) for url in urls] #1
status_codes = await asyncio.gather(*requests) #2
print(status_codes)
asyncio.run(main())
Shows the complete aiohttp pattern: create a client session, generate request coroutines, expand them into gather, and print the collected status codes. section 4.4
