liveBook · Manning

Data-driven chatbot design

As explained in Natural Language Processing in Action, Second Edition

A chatbot-design approach that selects or generates responses from statistical patterns learned from conversational data.

Data-driven chatbot design uses a large database of human conversations to find an input or phrase and reuse responses that people have given before. It replaces much of the tedious pattern-matching work with statistical matching, but exact character sequences remain fragile when users introduce typos, new wording, or unfamiliar expressions. section 1.5.3

Its practical strengths and weaknesses follow from how language is represented:

  • Reuse observed responses for familiar utterances instead of hand-coding every phrase. section 1.5.3
  • Handle variation by comparing meaning rather than relying only on exact character matches. section 1.5.3
  • Treat character-based distance cautiously: good and okay differ greatly as strings, while bad and bag differ slightly despite different meanings. section 1.5.3

Greeting recognition

A chatbot can search a conversation database for a greeting such as “Good morning” and reuse a response previously paired with it. A typo or novel phrasing may defeat exact lookup, motivating better methods for extracting meaning from text. section 1.5.3

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Defines database lookup and character-distance limitations, but defers deeper meaning extraction.
Examples
Uses greeting lookup, typo variation, and good-versus-okay and bad-versus-bag comparisons.
What you can do after
Evaluate when conversational response retrieval can outperform exact matching and where character-based similarity fails.
How it connects
Connects chatbot design with pattern matching, natural language processing, spelling correction, Jaccard distance, and Levenshtein distance.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.