liveBook · Manning

Asynchronous request

As explained in Designing AI Systems

An asynchronous request is an interaction in which the client submits work, receives an acknowledgment or job identifier, and retrieves progress or results later.

An asynchronous request lets a client submit work and receive a job identifier rather than wait for the complete result. The client can later check the job’s status and retrieve its result. This pattern suits AI workflows that take minutes or hours, when an open HTTP connection is vulnerable to server or proxy timeouts and client disconnections. section 2.3.4 section 2.6

A job-tracking interaction has three steps:

  • Submit the work. The API returns a job ID, a processing status, and a URL for checking the job; it may also provide an estimated duration. section 2.3.4
  • Check progress. A status response can report the current state, percentage complete, and active step. section 2.3.4
  • Retrieve the result. Once the status is completed, the response includes the workflow’s output. section 2.3.4

Research across documents

A research workflow searches a corpus, sends the documents to a reasoning model, and returns a report with sources. A request might take 10–15 minutes, so the client submits the topic, receives a job reference, and checks the status URL instead of holding the original connection open. section 2.3.4

Listing 2.13 A deep research workflow that requires asynchronous interaction
@workflow(name="deep_researcher", api_path="/research")    #A
def research_topic(topic, depth="comprehensive"):    #A
    documents = platform.data.search(
        index_name="research_corpus",
        query=topic,
        top_k=100
    )
 
    analysis = platform.models.chat(    #B
        model="o3",    #B
        messages=[    #B
            {"role": "system",    #B
             "content": f"Reference documents:\n{documents}"},    #B
            {"role": "user",    #B
             "content": f"Analyze these documents about {topic}"}]    #B
    )
 
    return {"research_report": analysis.content,
            "sources": [d.source for d in documents]}

The workflow searches for documents and returns a report with sources; its long execution motivates the asynchronous interface. section 2.3.4

{
  "job_id": "job_8x7f9k2m",
  "status": "processing",
  "status_url": "/jobs/job_8x7f9k2m",
  "estimated_duration": "10-15 minutes"
}

The initial response provides the job ID and status URL the client uses to track the request. section 2.3.4

{
  "job_id": "job_8x7f9k2m",
  "status": "completed",
  "result": {
    "research_report": "Recent advances in protein 
➥folding prediction have centered on...",
    "sources": [...]
  }
}

A completed status response carries the research report and its sources. section 2.3.4

Use a synchronous request when the operation completes in seconds and the client can reasonably wait. Use streaming when the client needs progressive token-by-token output; job tracking instead separates submission from later status and result retrieval. section 2.3.4 section 2.6

How Designing AI Systems covers it

How deep it goes
Shows a workflow and job-status responses but does not specify job-tracking internals.
Examples
Illustrates research across scientific documents with a job ID, progress response, and completed report with sources.
What you can do after
Evaluate when to use job tracking instead of a synchronous request for a long-running AI workflow.
How it connects
Contrasts synchronous requests and streaming responses with asynchronous job tracking.

Also explained in

Compare how 2 books explain asynchronous request →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.