Generative chatbot

Q: How deeply does Natural Language Processing in Action, Second Edition cover generative chatbot?

Works through prompt design, conversation memory, and observed failures; stops short of explaining model training internals.

As explained in Natural Language Processing in Action, Second Edition

A generative chatbot is a chatbot that dynamically produces conversational responses from learned language representations rather than selecting only from fixed responses.

A generative chatbot produces replies from a user’s message rather than choosing only from predefined answers. A pretrained language model can handle unexpected questions without a designer writing every response. That flexibility comes with a tradeoff: replies can be difficult to predict or control. section 12.4

A practical setup separates the model from the conversation around it:

Connect to a language model through an API; LangChain provides an interface for experimenting with different models. section 12.5.2
Write a prompt that states the bot’s role, the user’s message, and the response it should produce. For a math tutor, the prompt can specify how to ask a parent for permission before proceeding. section 12.5.2
Include conversation history when the next reply depends on earlier turns. A prompt receiving only the latest message did not reliably retain a user’s name. section 12.5.2

>>> class MathConversation():
...     def __init__(self, llm, prompt_string):
...         self.llm = llm
...         self.memory = \
...             ConversationBufferMemory(
...                 memory_key='history',
...                 ai_prefix='tutor',
...                 human_prefix="user")
...         self.convo_chain = ConversationChain(
...             llm=llm, memory=self.memory)
...         self.convo_chain.prompt = PromptTemplate(
...             input_variables=["history", "input"],
...             template=prompt_string)
...
...     def answer(self, user_input):
...         return self.convo_chain.predict(input=user_input)

This Python wrapper connects a prompt template to a LangChain conversation chain and memory, so each answer can use prior turns. section 12.5.2

A convincing reply can still be wrong

Asked to continue 2, 4, 6, 8, the tutor praised the answer “12” as correct while also saying the next number was 10. Check generated feedback against the actual task, not just its friendly tone. section 12.5.2

Do not rely on an unguarded model for sensitive or consequential exchanges. Ground or fine-tune it, add guardrails, and consider intent recognition to flag risky messages or assess proposed replies. Generative responses are useful for unusual cases, but fluency is not a reliability guarantee. section 12.4 section 12.5.2

How Natural Language Processing in Action, Second Edition covers it

Cover of Natural Language Processing in Action, Second Edition

liveBook manning.com

Read Natural Language Processing in Action, Second Edition on liveBook Get the book at manning.com

Related concepts

generative approachimplemented by · A conversational generation technique that produces responses dynamically instead of selecting them from a predefined set.
similarity searchuses · Similarity search is a retrieval technique that finds or ranks items by comparing their representations with a query according to a similarity or distance measure.
exact-match accuracymeasured by · Exact-match accuracy evaluates outputs by whether they exactly equal their references under specified comparison rules.
hybrid chatbotcontrasts with · A conversational system that combines generative language models with intent routing, rules, templates, search, or other conventional techniques.

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.