asyncio.as_completed runs a group of awaitables concurrently and returns an iterator of futures. Awaiting each future yields the result of whichever operation finishes next, so completed work can be processed without waiting for slower operations. Unlike asyncio.gather, it provides no deterministic result ordering. section 4.5 section 4.7
Use as_completed when:
- Results should reach users or downstream processing as soon as individual operations finish. section 4.5
- The completion order is more useful than the input order, such as with requests having widely different response times. section 4.5
- A group-level timeout is needed; pass timeout in seconds, but remember that unfinished tasks continue running and may be difficult to identify or cancel. section 4.5.1
Processing HTTP results as they arrive
Create several fetch_status coroutines with different delays, pass them to asyncio.as_completed, and print await finished_task inside a for loop. Two one-second requests produce results before the ten-second request, even though all three started concurrently. section 4.5
Listing 4.8 Using as_completed
import asyncio
import aiohttp
from aiohttp import ClientSession
from util import async_timed
from chapter_04 import fetch_status
@async_timed()
async def main():
async with aiohttp.ClientSession() as session:
fetchers = [fetch_status(session, 'https:/ / www.example .com', 1),
fetch_status(session, 'https:/ / www.example .com', 1),
fetch_status(session, 'https:/ / www.example .com', 10)]
for finished_task in asyncio.as_completed(fetchers):
print(await finished_task)
asyncio.run(main())
Shows the central pattern: build fetch coroutines, iterate over asyncio.as_completed, and await each completion-order future. section 4.5
