asyncio.to_thread runs a synchronous workflow function in a worker thread, allowing the asyncio event loop to accept and coordinate other HTTP requests while that function waits on downstream services. section 9.1.3 section 9.3.1
A synchronous request handler combines asyncio.to_thread with asyncio.wait_for. The timeout applies to the complete workflow, returns 504 when the budget expires, and aligns downstream gRPC deadlines with the remaining wall-clock budget. Separate worker processes provide separate event loops and thread pools; concurrent requests retain isolated service clients and context state. section 9.1.3 section 9.3.1
Listing 9.4 Synchronous request handler
import asyncio
import inspect
from fastapi import Request
from fastapi.responses import JSONResponse
def create_sync_handler(workflow_func, timeout):
sig = inspect.signature(workflow_func) #C
async def handle(request: Request):
body = await request.json()
trace_ctx = extract_trace_context( #A
request.headers #A
) #A
_trace_var.set(trace_ctx) #B
try:
sig.bind(**body) #C
except TypeError as e: #C
return JSONResponse( #C
{"error": str(e)}, #C
status_code=422 #C
) #C
try: #D
result = await asyncio.wait_for( #D
asyncio.to_thread( #D
workflow_func, **body #D
), #D
timeout=timeout #D
) #D
except asyncio.TimeoutError: #E
return JSONResponse( #E
{"error": "Request timed out"}, #E
status_code=504 #E
) #E
except Exception as e: #F
logger.exception( #F
"workflow request failed") #F
return JSONResponse( #F
{"error": "An internal error" #F
" occurred. Please" #F
" try again."}, #F
status_code=500 #F
) #F
return JSONResponse(result) #G
return handleThe request-handler listing shows the operational sequence: parse JSON, extract tracing context, validate arguments, dispatch the synchronous callable with asyncio.to_thread, and enforce a timeout. section 9.1.3
Synchronous workflow request
A POST request containing a patient question and identifier remains open while the workflow executes. A successful response returns structured JSON with the generated response and sources; longer-running workflows may require streaming or asynchronous mode. section 9.1.3
