In this chapter and the following chapter, we will cover some adaptation strategies for the deep NLP transfer learning modeling architectures that we have covered so far. In other words, given a pretrained architecture such as ELMo, BERT, or GPT, how can we carry out transfer learning more efficiently? We can employ several measures of efficiency here. We choose to focus on parameter efficiency, where the goal is to yield a model with the fewest parameters possible while suffering minimal reduction in performance. The purpose of this is to make the model smaller and easier to store, which would make it easier to deploy on smartphone devices, for instance. Alternatively, smart adaptation strategies may be required just to get to an acceptable level of performance in some difficult transfer cases.