1 Deep learning for NLP


This chapter covers

  • Taking a short road trip through machine learning applied to NLP
  • Learning about the historical roots of deep learning
  • Introducing vector-based representations of language

Language comes naturally to humans but has historically been hard for computers to grasp. This book addresses the application of recent, cutting-edge deep learning techniques to automated language analysis. In the last decade, deep learning has emerged as the vehicle of the latest wave in artificial intelligence (AI). Results have consistently redefined the state of the art for a plethora of data analysis tasks in a variety of domains. For an increasing number of deep learning algorithms, better- than-human (human-parity or superhuman) performance has been reported: for instance, speech recognition in noisy conditions and medical diagnosis based on images. Current deep learning–based natural language processing (NLP) outperforms all pre-existing approaches by a large margin. What exactly makes deep learning so suitable for these intricate analysis tasks, in particular language processing? This chapter presents some of the background necessary to answer this question and guides you through a selection of important topics in machine learning for NLP.

1.1 A selection of machine learning methods for NLP

1.1.1 The perceptron

1.1.2 Support vector machines

1.1.3 Memory-based learning

1.2 Deep learning

1.3 Vector representations of language

1.3.1 Representational vectors

1.3.2 Operational vectors

1.4 Vector sanitization

1.4.1 The hashing trick

1.4.2 Vector normalization