Chapter 1. Packets of thought (NLP overview)

This chapter covers

  • What natural language processing (NLP) is
  • Why NLP is hard and only recently has become widespread
  • When word order and grammar is important and when it can be ignored
  • How a chatbot combines many of the tools of NLP
  • How to use a regular expression to build the start of a tiny chatbot

You are about to embark on an exciting adventure in natural language processing. First we show you what NLP is and all the things you can do with it. This will get your wheels turning, helping you think of ways to use NLP in your own life, both at work and at home.

Then we dig into the details of exactly how to process a small bit of English text using a programming language like Python, which will help you build up your NLP toolbox incrementally. In this chapter, you’ll write your first program that can read and write English statements. This Python snippet will be the first of many you’ll use to learn all the tricks needed to assemble an English language dialog engine—a chatbot.

1.1. Natural language vs. programming language

1.2. The magic

1.3. Practical applications

1.4. Language through a computer’s “eyes”

1.5. A brief overflight of hyperspace

1.6. Word order and grammar

1.7. A chatbot natural language pipeline

1.8. Processing in depth

1.9. Natural language IQ