liveBook · Manning

Fallback

As explained in LLMs in Production

A fallback response is a predefined response returned when processing a user request fails.

When a chatbot request fails because of connectivity, server load, or another error, a fallback returns a predefined response instead of leaving the user without feedback. section 8.1.3

Use a fallback to preserve a one-to-one relationship between user queries and LLM responses in chat history. Give clear guidance, such as asking the user to try again later. section 8.1.3

A practical chatbot interaction design should also:

  • Disable Submit while receiving a response to avoid asynchronous conversations and out-of-order chat history. section 8.1.3
  • Offer Retry to resend the last query and replace an unfavorable response. section 8.1.3
  • Offer Stop to interrupt long or incorrect responses and limit output-token costs. section 8.1.3

How LLMs in Production covers it

How deep it goes
Defines fallback behavior and its chat-history rationale, but does not specify a concrete error-handling implementation.
Examples
Uses connection failures, overwhelmed servers, and instructions to try again in a few minutes as concrete cases.
What it assumes
Assumes familiarity with chatbot requests, LLM responses, and stored chat history.
What you can do after
Operate/configure a chatbot that returns guidance after failed requests and maintains clean conversational history.
How it connects
Connects fallback responses with chat history, retry controls, stop controls, and asynchronous conversation handling.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 15 books explain fallback →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.