liveBook · Manning

Polling

As explained in Designing AI Systems

Polling is the repeated checking of an endpoint or resource until a desired state or result is available.

Polling repeatedly checks a resource until a desired state or result is available. For a long-running workflow, it lets a client receive an HTTP 202 Accepted response immediately, then check a status URL rather than hold the original request open. section 9.1.5

The runtime creates a job record, starts the workflow in the background, and returns a job ID and status URL. The Workflow Service owns the record and serves the polling endpoint from its database; the workflow container executes the function and writes progress, results, or errors. This separation also gives crash recovery access to the original input, assigned endpoint, and processing state. section 9.1.5 section 9.2.1

A polling client reads GET /jobs/{job_id} until the job reaches a terminal state:

  • While processing, the response can include a progress message. section 9.2.1
  • When completed, the client retrieves the result; when failed or cancelled, it handles an error. section 9.4.3
  • If the job is still running, the client waits before checking again. The SDK helper uses a two-second default interval. section 9.4.3

Long-running research

A research request returns a job ID and status URL. Successive polls report document retrieval and report formatting while the job is processing; a later response returns the completed research report and sources. section 9.1.5 section 9.2.1

Listing 9.10 Polling endpoint served by the Workflow Service
from fastapi import FastAPI
from fastapi.responses import JSONResponse
 
jobs_app = FastAPI()    #A
 
@jobs_app.get("/jobs/{job_id}")
async def get_job_status(job_id: str):
    row = jobs_store.execute(    #B
        """SELECT status, progress_message,
                  result_json, error,
                  created_at, updated_at
           FROM jobs WHERE job_id = %s""",
        (job_id,)
    )
 
    if not row:    #C
        return JSONResponse(    #C
            {"error": "Job not found"},    #C
            status_code=404
        )
 
    response = {    #D
        "job_id": job_id,    #D
        "status": row.status,    #D
        "created_at": row.created_at,    #D
        "updated_at": row.updated_at,    #D
    }
 
    if row.progress_message:    #E
        response["progress"] = (
            row.progress_message
        )
    if row.status == "completed":    #F
        response["result"] = json.loads(
            row.result_json
        )
    if row.status == "failed":    #G
        response["error"] = row.error
 
    return JSONResponse(response)

The endpoint reads the job record by ID and returns a 404 when it cannot find the job. section 9.2.1

For workflow composition, the SDK can hide polling from the parent. When a child returns 202, the SDK polls until completion and returns the result or raises an error. The parent continues to use the same call interface if the child changes from synchronous to asynchronous execution. section 9.4.3

Listing 9.20 Polling and stream consumption helpers for workflow-to-workflow calls
def _poll_until_complete(    #A
    self, job_id, interval=2.0
):
    while True:
        status = requests.get(
            f"{self._gateway_url}"
            f"/jobs/{job_id}",
            headers=self._trace_headers()
        ).json()
 
        if status["status"] == "completed":
            return status["result"]
        if status["status"] == "failed":
            raise WorkflowCallError(
                status["error"]
            )
        if status["status"] == "cancelled":
            raise WorkflowCallError(
                "Job was cancelled"
            )
 
        time.sleep(interval)    #B
 
def _consume_stream(self, response):    #C
    chunks = []
    for line in response.iter_lines():
        if line.startswith("data: "):
            chunk = json.loads(
                line[len("data: "):]
            )
            if not chunk.get("done"):
                chunks.append(chunk)
    return chunks

The polling helper shows the terminal-state checks and the delay between requests. section 9.4.3

How Designing AI Systems covers it

How deep it goes
Works through job storage, the status endpoint, and terminal-state handling, without specifying a client timeout.
Examples
Shows a research workflow progressing from document retrieval and report formatting to a completed report and sources.
What you can do after
Apply it via platform.workflows.call, which polls asynchronous child workflows and returns their results or raises errors.
How it connects
Connects asynchronous workflows with job persistence, crash recovery, workflow composition, and streaming responses.

Also explained in

Compare how 10 books explain polling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.