chapter seven

7 Attention

 

This chapter covers:

  • How to implement attention in multi-layer perceptrons (static attention) and LSTMs (temporal attention).
  • How attention may help to improve performance of a deep learning model.
  • How attention helps to explain the outcomes of models, by highlighting attention patterns connected to input data.

The following picture displays the chapter organization:

Figure 7.1. Chapter organization.
mental model chapter7 all

7.1  Neural attention

mental model chapter7 intro

7.2  Data

7.3  Static attention: MLP

7.4  Temporal attention: LSTM

7.4.1  Experiments

7.5  Summary

7.6  Further reading