Server-Sent Events (SSE) deliver a stream of events from a server to a client over HTTP. For AI responses, SSE lets a conversational interface display text as it is generated instead of waiting for the complete answer. It differs from a synchronous response returned all at once and from an asynchronous job whose status is polled later. section 2.3.4 section 9.1.4
A client can request a stream with Accept: text/event-stream. The response uses Content-Type: text/event-stream and sends successive data fields, each containing a JSON value. A final event can carry a completion flag and session ID so the client knows the workflow has finished. section 2.3.4 section 9.1.4
HTTP/1.1 200 OK
Content-Type: text/event-stream
data: {"token": "Please"}
data: {"token": " bring"}
data: {"token": " your"}
data: {"token": " insurance"}
data: {"token": " card"}
data: {"done": true, "session_id": "sess-456"}Inspect the response headers, token events, and final completion event. section 9.1.4
A streaming workflow calls platform.models.chat_stream(), iterates over the returned chunks, and yields each token. Setting response_mode="stream" selects a runtime handler that turns yielded values into SSE events. section 9.1.4
Listing 9.5 A streaming workflow that yields tokens progressively
@workflow(
name="patient_intake_assistant",
api_path="/patient-assistant",
response_mode="stream", #A
timeout_seconds=30
)
def handle_patient_question(question, patient_id):
platform = GenAIPlatform()
session = platform.sessions.get_or_create(
patient_id
)
history, _ = platform.sessions.get_messages(
session.session_id
)
stream = platform.models.chat_stream( #B
model="gpt-4o", #B
system_prompt_name="patient_intake_assistant",
messages=[*history,
{"role": "user",
"content": question}]
)
for chunk in stream: #C
yield {"token": chunk.token} #C
yield {"done": True,
"session_id": session.session_id}See the workflow configuration, token yields, and completion signal together. section 9.1.4
The runtime validates request arguments and propagates trace context before iterating over the workflow generator. A per-chunk timeout detects a stalled model stream; a sentinel marks generator exhaustion without letting StopIteration escape from the task. section 9.1.4
Listing 9.6 The runtime server's SSE handler for streaming workflows
from fastapi.responses import StreamingResponse
_DONE = object() #A
def create_stream_handler(workflow_func, timeout):
sig = inspect.signature(workflow_func)
async def handle(request: Request):
body = await request.json()
trace_ctx = extract_trace_context( #B
request.headers #B
) #B
_trace_var.set(trace_ctx) #C
try:
sig.bind(**body) #D
except TypeError as e: #D
return JSONResponse( #D
{"error": str(e)}, #D
status_code=422 #D
) #D
async def event_stream():
gen = workflow_func(**body) #E
try: #F
while True:
chunk = await asyncio.wait_for( #G
asyncio.to_thread( #G
next, gen, _DONE #G
), #G
timeout=timeout #G
) #G
if chunk is _DONE: #H
break #H
data = json.dumps(chunk)
yield f"data: {data}\n\n"
except asyncio.TimeoutError: #I
yield f"data: {json.dumps({'error': 'Stream timed out'})}\n\n"
except Exception: #J
logger.exception("stream failed") #J
yield f"data: {json.dumps({'error': 'Stream interrupted'})}\n\n"
return StreamingResponse( #K
event_stream(), #K
media_type="text/event-stream" #K
) #K
return handleInspect the handler's request validation and generator iteration. section 9.1.4
SSE can also sit at both ends of a larger streaming pipeline. Provider adapters normalize differing provider stream formats; the Model Service forwards chunks through gRPC streaming; and the API gateway converts that stream to SSE for external clients. Following these boundaries helps locate streaming latency. section 3.3.1



