The field of NLP has seen rapid progress in the past few years. Specifically, the advent of the Transformer and pretrained language models such as BERT have completely changed the landscape of the field and how practitioners build NLP applications. This part of the book will help you catch up with these latest developments.
Chapter 6 introduces sequence-to-sequence models, an important class of models that will enable you to build more complex applications such as machine translation systems and chatbots. Chapter 7 discusses another type of popular neural network architecture, convolutional neural networks (CNNs).
Chapters 8 and 9 are arguably the most important and exciting chapters of this book. They cover the Transformer and transfer learning methods (such as BERT) respectively. We’ll demonstrate how to build advanced NLP applications such as high-quality machine translation and spell-checkers, using those technologies.