liveBook · Manning

Finish reason

As explained in Designing AI Systems

A finish reason is a response field indicating why a model stopped generating output, such as natural completion, a limit, a tool call, filtering, or an error.

A finish reason is a response field that records why model generation stopped. Typical values are stop for natural completion, length for a token limit, tool_calls for requested function execution, content_filter for filtered output, and error for a mid-stream failure. Consumers should treat the reason as part of the response contract rather than infer termination from the text. section 3.3.2 section 3.1.6

Handle the field according to response mode:

  • For non-streaming responses, inspect finish_reason alongside content, tool_calls, usage, model, and provider to determine what happened and record the result accurately. section 3.1.6
  • For streaming responses, expect finish_reason only on the final ChatChunk; continue consuming chunks until a chunk supplies the field. section 3.3.2
  • If the reason is error, preserve the tokens already delivered and expose an error state instead of discarding the partial response. section 3.3.3
  • Do not assume streaming fallback can continue transparently after failure: recovery usually applies before streaming begins because changing providers mid-response changes its style. section 3.3.3

Streaming termination

A consumer appends each incoming token to the displayed response. When the final chunk contains finish_reason = "length", it marks the response as limit-truncated; when it contains "error", it keeps the partial text but reports the failed stream. section 3.3.2 section 3.3.3

Listing 3.10 ChatChunk message for streaming responses
message ChatChunk {
  string token = 1;    #A
  string model = 2;    #B
  optional string finish_reason = 3;    #C
  optional TokenUsage usage = 4;
}

Use the ChatChunk schema to see finish_reason as an optional field paired with streamed tokens, model identity, and final usage. section 3.3.2

How Designing AI Systems covers it

How deep it goes
Defines termination values and final-chunk behavior, then addresses partial output and fallback limits without implementing provider-specific adapters.
Examples
Uses ChatChunk and ChatResponse schemas plus partial-output handling for mid-stream errors.
What you can do after
Diagnose completed and failed generations by inspecting finish_reason and preserving valid partial output during streaming.
How it connects
Connects finish reason to streaming responses, token usage, tool calls, content filtering, model service, and fallback behavior.

Also explained in

Compare how 2 books explain finish reason →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.