8 Multitask learning
This chapter covers
- Deep multitask learning for NLP: the joint learning of NLP tasks, with the aim of improving on each subtask by learning them jointly
- Implementing hard parameter sharing for multitask learning
- Implementing soft parameter sharing for multitask learning
- Combining hard and soft parameter sharing into mixed parameter sharing
You will apply different approaches of multitask learning to practical NLP problems. In particular, we will apply multitask learning to three datasets:
- Two sentiment datasets, consisting of consumer product reviews and restaurant reviews.
- The Reuters topic dataset.
- A part-of-speech and named entity tagging dataset.
Figure 8.1. An introduction to multitask learning. Classifier performance improvement by learning several tasks in one go.
Multitask learning is concerned with learning several things at the same time. An example would be to learn both part of speech tagging and sentiment analysis at the same time. Or learning two topic taggers in one go. Why would that be a good idea? Ample research has demonstrated, for quite some time already, that multitask learning improves the performance on certain tasks separately. This gives rise to the following application scenario: