7 Convolutional neural networks

 

This chapter covers

  • Solving text classification by detecting patterns
  • Using convolutional layers to detect patterns and produce scores
  • Using pooling layers to aggregate the scores produced by convolution
  • Building a convolutional neural network (CNN) by combining convolution and pooling
  • Building a CNN-based text classifier using AllenNLP

In previous chapters, we covered linear layers and RNNs, two main neural network architectures commonly used in NLP. In this chapter, we introduce another important class of neural networks called convolutional neural networks (CNNs). CNNs have different characteristics than RNNs that make them suitable for NLP tasks where detecting linguistic patterns is important, such as text classification.

7.1 Introducing convolutional neural networks (CNNs)

This section introduces convolutional neural networks (CNNs), another type of neural network architecture that operates in a different way from how RNNs work. CNNs are particularly good at pattern-matching tasks and are increasingly popular in the NLP community.

7.1.1 RNNs and their shortcomings

7.1.2 Pattern matching for sentence classification

7.1.3 Convolutional neural networks (CNNs)

7.2 Convolutional layers

7.2.1 Pattern matching using filters

7.2.2 Rectified linear unit (ReLU)

7.2.3 Combining scores

7.3 Pooling layers

7.4 Case study: Text classification

7.4.1 Review: Text classification

7.4.2 Using CnnEncoder

7.4.3 Training and running the classifier

Summary