Token streaming delivers generated text in chunks as it becomes available, rather than waiting for a complete response. In a chatbot, this creates a live-typing experience and makes an individual request feel more responsive. It does not, by itself, make the answer more accurate. section 9.2.1 section 9.0
Choose the display mode to match the task. Streaming suits conversational replies, but unfinished sentences or changes of direction can confuse users. For a legal brief, research summary, or formal letter, full-response generation can present a coherent answer at once and avoid exposing partial misinformation. section 9.2.1
from openai import AsyncOpenAI #1
import asyncio #2
client = AsyncOpenAI() #3
async def stream_response(prompt):
response = await client.chat.completions.create(
model="gpt-5.4-mini", #4
messages=[{"role": "user", "content": prompt}],
stream=True #5
)
async for chunk in response:
if chunk.choices and chunk.choices[0].delta.content:
partial = chunk.choices[0].delta.content
print(partial, end="", flush=True) #6
async def main():
await stream_response("How can I reset my password?") #7
asyncio.run(main()) #8
The Python example sets stream=True, asynchronously iterates over response chunks, checks for content in each delta, and prints each piece immediately. section 9.2.1
Streaming is a per-request presentation choice, not a substitute for traffic management. An e-commerce service can use a request flag to stream selected replies while returning other answers as single chunks. It can separately queue and batch requests, cache similar answers, and route complex queries to a more capable model. section 9.2.5 section 9.2
