Chapter 11. Neural networks that understand language: king – man + woman == ?
In this chapter
- Natural language processing (NLP)
- Supervised NLP
- Capturing word correlation in input data
- Intro to an embedding layer
- Neural architecture
- Comparing word embeddings
- Filling in the blank
- Meaning is derived from loss
- Word analogies
“Man is a slow, sloppy, and brilliant thinker; computers are fast, accurate, and stupid.”
John Pfeiffer, in Fortune, 1961
Up until now, we’ve been using neural networks to model image data. But neural networks can be used to understand a much wider variety of datasets. Exploring new datasets also teaches us a lot about neural networks in general, because different datasets often justify different styles of neural network training according to the challenges hidden in the data.
We’ll begin this chapter by exploring a much older field that overlaps deep learning: natural language processing (NLP). This field is dedicated exclusively to the automated understanding of human language (previously not using deep learning). We’ll discuss the basics of deep learning’s approach to this field.