liveBook · Manning

Synchronous call

As explained in Designing AI Systems

Synchronous execution is an execution model in which the caller waits for an operation to complete before receiving its result.

A synchronous call waits for an operation to finish before returning its result. It fits fast, predictable work, such as image classification or a tool call to a scheduling API. A caller waiting seconds can often proceed without a separate job-tracking interface; a caller waiting minutes or hours needs a different interaction pattern. section 2.3.4 section 6.3.4

Image categorization

A client POSTs an image URL to a workflow and waits while the vision model categorizes it. The workflow returns the category in the response; the illustrated model call takes roughly two to three seconds. section 2.3.4

Listing 2.12 An image categorizer workflow that works with synchronous interaction
@workflow(name="image_categorizer", 
➥api_path="/categorize-image")    #A
def categorize_image(image_url):    #A
   result = platform.models.chat(    #B
      model="gpt-4o",    #B
      messages=[    #B
         {    #B
            "role": "user",    #B
            "content": [    #B
               {    #B
                  "type": "text",    #B
                  "text": "Categorize this 
➥product image into one of these categories: 
➥outerwear, sportswear, formal wear, casual, 
➥accessories"    #B
               },    #B
               {    #B
                   "type": "image_url",    #B
                   "image_url": 
➥{"url": image_url}    #B
                }    #B
             ]    #B
         }    #B
      ]    #B
   )    #B
   
   return {"category": result.content}    #B

The workflow exposes an image-categorization endpoint and returns the model result directly. section 2.3.4

For external integrations, the Tool Service can call the external system, wait, and return the result to the application. Most reads and simple writes fit this path. For slower operations, asynchronous execution returns a task identifier immediately and lets the application retrieve the result later, preserving conversation responsiveness. section 6.3.4

A production synchronous workflow handler needs more than a direct function call:

  • Validate the JSON body against the function signature and return a descriptive 422 for missing or unexpected parameters. section 9.1.3
  • Extract distributed trace context from request headers so subsequent platform calls join the same trace without changing the workflow function signature. section 9.1.3
  • Apply a whole-workflow timeout and return 504 when it expires; propagate the remaining time as deadlines to downstream calls. section 9.1.3
  • Run synchronous workflow code in a separate thread so it does not block the event loop from handling other requests. section 9.1.3
Listing 9.4 Synchronous request handler
import asyncio
import inspect
from fastapi import Request
from fastapi.responses import JSONResponse
 
def create_sync_handler(workflow_func, timeout):
    sig = inspect.signature(workflow_func)    #C
 
    async def handle(request: Request):
        body = await request.json()
        trace_ctx = extract_trace_context(    #A
            request.headers    #A
        )    #A
        _trace_var.set(trace_ctx)    #B
 
        try:
            sig.bind(**body)    #C
        except TypeError as e:    #C
            return JSONResponse(    #C
                {"error": str(e)},    #C
                status_code=422    #C
            )    #C
 
        try:    #D
            result = await asyncio.wait_for(    #D
                asyncio.to_thread(    #D
                    workflow_func, **body    #D
                ),    #D
                timeout=timeout    #D
            )    #D
        except asyncio.TimeoutError:    #E
            return JSONResponse(    #E
                {"error": "Request timed out"},    #E
                status_code=504    #E
            )    #E
        except Exception as e:    #F
            logger.exception(    #F
                "workflow request failed")    #F
            return JSONResponse(    #F
                {"error": "An internal error"    #F
                          " occurred. Please"    #F
                          " try again."},    #F
                status_code=500    #F
            )    #F
 
        return JSONResponse(result)    #G
    return handle

The handler illustrates signature validation, trace-context setup, and bounded execution of synchronous workflow code. section 9.1.3

Configure expected latency, maximum wait time, and retry safety at the tool level. An idempotency flag can allow the platform to retry safely rather than leaving each application to hardcode its own policy. section 6.3.4

How Designing AI Systems covers it

How deep it goes
Explains request handling through validation, tracing, timeouts, thread execution, and downstream deadlines.
Examples
Illustrates image categorization, a scheduling API, and a synchronous workflow handler.
What you can do after
Evaluate when synchronous execution fits and configure a workflow handler with validation, tracing, and timeouts.
How it connects
Connects synchronous execution to asynchronous execution, streaming, distributed tracing, timeouts, and idempotency.

Also explained in

Compare how 12 books explain synchronous call →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.