A chatbot may handle store hours with a fixed answer, then stumble when a customer asks an unexpected question. Generative modeling offers another option: create a response from the customer’s request rather than select only from predefined replies. It can answer users directly or help a team prepare better responses before deployment. section 1.1.3 section 1.2
A generative model creates new content based on its input and learned patterns. In a text-based conversational system, a large language model (LLM) generates a sequence by repeatedly predicting what comes next. A plausible answer is not proof that the model understood the request or checked its facts. section 1.2.1 section 4.1.4
When fixed answers are too narrow
Suppose a bakery bot uses an intent classifier for routine questions and rules for cake orders. Customers also ask for dinner recipes, with no consistent type of dish. The bot can keep its controlled ordering flow while routing detected recipe requests to an LLM for a generated answer. This is a hybrid choice, not a reason to replace every existing response. section 2.3 section 2.4
Choose where generation earns its place:
- For customer-facing answers, retrieval-augmented generation (RAG) can generate an answer from knowledge-base search results instead of relying only on the model’s learned patterns. section 1.1.3 section 1.2.1
- For build-time work, generate candidate training utterances, draft or simplify dialogue, and simulate users testing a process flow. Review outputs before adopting them. section 7.0 section 10.0
- For handoff, generate a conversation summary when a transcript and a clear definition of a good summary are available. section 12.3
Figure 4.4 Traditional classification models use supervised learning to predict one of several predefined classifications. They look for the intent, or meaning, of a user input. Generative models use decoding transformers to create a text completion. They predict the next sequence of tokens (loosely, words or characters) that are most likely to occur after the user input.
The figure contrasts selecting a predefined classification with generating a text completion. That difference helps determine whether a task needs a controlled intent and response or a newly composed answer. section 4.1.4
Start with the cost of a wrong answer. An LLM can produce fluent misinformation, and the organization remains responsible for what its chatbot says. Test several suitable models, prompts, and settings rather than trusting one demonstration; apply guardrails when outputs reach users. Document where generation runs, which sources it draws on, and what success should look like. Then measure whether responses are correct and useful against representative user requests, and monitor results as the solution changes. section 1.2.3 section 4.3.2 section 4.6 section 4.2
