This chapter covers:
- Deep multitask learning for NLP: the joint learning of NLP tasks, with the aim of improving on each subtask by learning them jointly.
- How to implement hard parameter sharing for multitask learning.
- How to implement soft parameter sharing for multitask learning.
- How to combine hard and soft parameter sharing into mixed parameter sharing.
You will apply different approaches of multitask learning to practical NLP problems. In particular, we will apply multitask learning to three datasets:
- Two sentiment datasets, consisting of consumer product reviews and restaurant reviews.
- The Reuters topic dataset.
- A part-of-speech and named entity tagging dataset.
The following picture displays the chapter organization:
Figure 8.1. Chapter organization.
Figure 8.2. This section will introduce you to multitask learning.
Multitask learning is concerned with learning several things at the same time. An example would be to learn both part of speech tagging and sentiment analysis at the same time. Or learning two topic taggers in one go. Why would that be a good idea? Ample research has demonstrated, for quite some time already, that multitask learning improves the performance on certain tasks separately. This gives rise to the following application scenario: