liveBook · Manning

Asynchronous programming

As explained in Designing AI Systems

Asynchronous execution is an execution model in which a request returns before its operation completes, allowing processing to continue independently in the background.

Asynchronous programming lets a request return before its operation finishes, keeping conversations and services responsive while long-running work continues in the background. A service accepts the request, creates a task or job identifier, and exposes completion through polling or a callback. section 6.3.4 section 9.1.5

A production asynchronous pattern typically follows these steps:

  • Accept the request and record its input for recovery. section 9.1.5
  • Return an identifier immediately, commonly with HTTP 202 Accepted and a status URL. section 9.1.5
  • Run the operation as a background task, allowing the service to handle other requests. section 9.1.5
  • Track progress and completion through polling or callbacks, and apply timeout, retry, and idempotency policies at the tool boundary. section 6.3.4 section 9.5.2

Insurance verification

A tool service can call execute_async with patient and insurance data, receive task-789 with status pending, then poll get_task every two seconds or register a completion webhook. The assistant can continue the conversation instead of waiting for backend verification. section 6.3.4

Figure 6.7 Synchronous and asynchronous execution patterns manage the unpredictable latency of external integrations. For fast, predictable operations, the Synchronous Pattern allows the Tool Service to block the request, wait for external processing, and return the result directly to the application. Conversely, for slow or complex operations, such as insurance verification, the Asynchronous Pattern prevents the AI conversation from stalling. The platform immediately returns a task ID while processing the request in the background, allowing the application to use callbacks or polling to retrieve the result once completed. This dual-pattern approach ensures the AI assistant remains responsive and maintains a seamless user experience, regardless of backend system delays.

Compare synchronous blocking with asynchronous task-based execution, including task IDs, background processing, polling, and callbacks. section 6.3.4

How Designing AI Systems covers it

How deep it goes
Explains task and job lifecycles, polling, callbacks, progress, recovery, and HTTP handling, but defers detailed message schemas.
Examples
Uses insurance verification, deep document research, data analysis, batch processing, task polling, callbacks, and HTTP 202 responses.
What it assumes
Assumes familiarity with external services, HTTP requests, workflow functions, and tool or job-service boundaries.
What you can do after
Operate long-running tools and workflows through job identifiers, background execution, status polling, callbacks, and progress management.
How it connects
Connects synchronous execution, background tasks, polling, callbacks, retries, idempotency, and workflow services.

Also explained in

Compare how 37 books explain asynchronous programming →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.