liveBook · Manning

Conversational ai

As explained in Natural Language Processing in Action, Second Edition

A dialog engine is a software system that manages conversational interaction and generates responses to user statements.

Conversational AI lets software respond to people in natural language. Its dialog engine, or chatbot, takes a statement, works out what the person may want, and produces a reply. A voice assistant adds speech input and output around that process; a human-sounding voice is not, by itself, evidence that the system understands the conversation. section 1.2.2 section 12.7.4

A useful chatbot does more than recognize isolated words. It must track conversation state, recognize user intent, and choose text that helps the user reach a goal. Those goals matter: a customer-service bot that delays access to a person may serve its operator while failing its user. section 12.9 section 12.0

Start with a greeting rule

A minimal Python bot can split a message into words and check its first word: `greetings = "Hi Hello Greetings".split(); words = "Hello Joshua".split(); reply = "Hello!" if words[0] in greetings else "I don't recognize that greeting."` This illustrates an intent recognizer with two outcomes: greeting and unknown. It also shows the weakness of a hand-written rule: another phrasing needs another rule or a more flexible recognizer. section 1.5.1 section 1.5

Response strategies can be combined rather than treated as rival definitions of a chatbot:

  • Rules and regular expressions match recognizable phrases, extract information, and trigger scripted responses. They offer control but require someone to anticipate ways users will speak. section 1.4.2 section 1.5
  • Search retrieves a relevant response from stored text or past conversations. Exact-string lookup is brittle: a typo or new phrasing can defeat it even when the meaning is familiar. section 1.12 section 1.5.3
  • Language models can generate candidate replies; templates and search are other ways to generate them. A separate decision step can select a response suited to the conversational goal. section 1.8
Figure 1.7 Chatbot recirculating (recurrent) pipeline
figure

The recirculating pipeline places a reply inside an ongoing exchange. Parsing extracts features; analysis scores them; generation proposes responses; decision selects one; and a database retains conversation history, user information, and knowledge. section 1.8

The same tools can support a virtual assistant, a team-operations bot, a healthcare information service, or a customer-service interface. The application determines what the bot needs to know and what a successful conversation looks like. Speech-to-text and text-to-speech can add a voice, but the dialog engine still needs to handle the words between them. section 12.1 section 12.7.4

Do not mistake fluent output for reliable cooperation. Rule-based approaches remain useful when behavior must be controlled; a language model alone is neither fully explainable nor controllable. Design around users’ likely needs, maintain context, and evaluate replies by whether they help people achieve their goals. section 12.9

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Builds from a two-intent Python recognizer to response strategies and a five-subsystem pipeline, while treating speech processing largely as an external wrapper.
Examples
Uses a Python greeting recognizer, customer-service bots, virtual assistants, and a diagram of the recirculating chatbot pipeline.
What it assumes
Assumes basic Python for the runnable recognizer but introduces intent categories through greeting and unknown outcomes.
What you can do after
Build it from scratch by assembling recognition, response generation, decision, and memory components around a user's conversational goal.
How it connects
Connects intent recognition, regular expressions, semantic search, language models, conversation state, and speech-to-text.

Also explained in

Compare how 20 books explain conversational ai →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.