chapter one

1 Machines that read and write (NLP overview)

 

This chapter covers

  • The power of human language
  • How natural language processing (NLP) is changing society
  • The kinds of NLP tasks that machines can now do well
  • Why unleashing the NLP genie is profitable … and dangerous
  • How to start building a simple chatbot
  • How NLP technology is programming itself and making itself smarter

Words are powerful. They can change minds. And they can change the world. Natural language processing puts the power of words into algorithms. In fact, those algorithms are changing your world right before your eyes. You are about to see how the majority of the words and ideas that enter your mind are filtered and generated by NLP.

Imagine what you would do with a machine that could understand and act on every word it reads on the Internet? Imagine the information and knowledge you’d be able to harvest and profit from. NLP promises to create the second information revolution by turning vast amounts of unstructured data into actionable knowledge and understanding.

Early on, Big Tech discovered this power of NLP to harvest knowledge from natural language text. They use that knowledge to affect our behavior and our minds in order to improve their bottom line.[1] Governments too are waking up to the impact NLP has on culture, society and humanity. Some advanced liberal democracies are attempting to free your mind by steering business towards sustainable and ethical uses for NLP.

1.1 Programming language vs. natural language

1.1.1 Natural Language Understanding (NLU)

1.1.2 Natural Language Generation (NLG)

1.1.3 Plumbing it all together for positive impact

1.2 The magic

1.2.1 Language and thought

1.2.2 Machines that converse

1.2.3 The math

1.3 Applications

1.3.1 Processing programming languages with NLP

1.4 Language through a computer’s "eyes"

1.4.1 The language of locks

1.4.2 Regular expressions

1.5 A simple chatbot

1.6 Keyword-based greeting recognizer