8 Multitask learning

 

This chapter covers

  • Understanding deep multitask learning for NLP
  • Implementing hard, soft, and mixed parameter sharing for multitask learning

In this chapter, we apply different multitask learning approaches to practical NLP problems. In particular, we apply multitask learning to three datasets:

  • Two sentiment datasets consisting of consumer product reviews and restaurant reviews
  • The Reuters topic dataset
  • A part-of-speech and named-entity tagging dataset

8.1 Introduction to multitask learning

Multitask learning is concerned with learning several things at the same time (figure 8.1). An example is learning both part-of-speech tagging and sentiment analysis simultaneously or learning two topic taggers in one go. Why would that be a good idea? For quite some time, ample research has demonstrated that multitask learning improves performance on certain separate tasks. This gives rise to the following application scenario.

Figure 8.1 An introduction to multitask learning. Classifier performance improves by learning several tasks in one go.
Scenario: Multitask learning

You are training classifiers on a number of NLP tasks, but the performance is disappointing. It turns out your tasks can be decomposed into separate subtasks. Can multitask learning be applied here, and, if so, does performance improve on the separate tasks when they are learned together?

8.2 Multitask learning

8.3 Multitask learning for consumer reviews: Yelp and Amazon

8.3.1 Data handling

8.3.2 Hard parameter sharing

8.3.3 Soft parameter sharing

8.3.4 Mixed parameter sharing

8.4 Multitask learning for Reuters topic classification

sitemap