liveBook · Manning

Finish reason

As explained in Generative AI in Action

A finish reason is an API response field indicating why model generation ended or why the API returned the output it did.

A finish reason is the response field that identifies why text generation ended or why an API returned a particular output. Tracking it supports debugging, completeness checks, and production handling. section 3.4.2 section 3.5

Interpret the principal finish_reason values as follows:

  • stop means generation completed normally or ended at a supplied stop sequence. section 3.4.2
  • length means max_tokens or the token limit stopped the output, potentially leaving it incomplete. section 3.4.2
  • function_call means the model chose to call a function. section 3.4.2
  • content_filter means harmful content triggered filtering and may produce a null or empty response. section 3.2.2 section 3.4.2

Handling a streaming response

During streaming, intermediate messages usually carry finish_reason: null while each message contributes newly generated text. The final message reports stop or length, so client logic should wait for that message before deciding whether the response completed successfully. section 3.3.1

Listing 3.5 Output showing null response
$ python .\petsalon.py 
Prompt:Suggest three names for a new pet salon business. The generated 
       name ideas should evoke positive emotions and the following key 
       features: Professional, friendly, Personalized Service.
Response:                                    #1
{
  "choices": [
    {
      "finish_reason": "content_filter",      #2
      "index": 0,
      "logprobs": null,
      "text": "",
      "content_filter_results"={...}
    }
  ],
  "created": 1689006467,
  "id": "cmpl-7ao1jIACW1v8mYH879EE1trbT9Ua6",
  "model": "gpt35",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 31,
    "prompt_tokens": 32,
    "total_tokens": 63
  }
}

Inspect a null-output response where content_filter identifies the reason and the completion text is empty. section 3.2.2

How Generative AI in Action covers it

How deep it goes
Defines four principal values and explains streaming nulls, but defers detailed content-safety treatment.
Examples
Uses pet-salon completion JSON, filtered empty output, and streamed token messages ending with stop.
What it assumes
Assumes familiarity with API completion responses, choices, generated text, and token limits.
What you can do after
Diagnose/debug with it by distinguishing normal, truncated, filtered, function-calling, and still-streaming responses.
How it connects
Connects finish reasons to completions, chat completions, streaming, stop sequences, token limits, function calling, and content filtering.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Generative AI in Action. The section links point to the parts of the book it draws on.