9 Tailoring models with model adaptation and fine-tuning

 

This chapter covers

  • Basics of model adaptation and its advantages
  • How to train an LLM
  • How to fine-tune an LLM using both SDK and GUI
  • Best practices for evaluation criteria and metrics for fine-tuned LLMs
  • How to deploy a fine-tuned model for inference
  • Gaining insight into key model adaptation techniques

As we explore the intricate world of large language models (LLMs), a key aspect that stands at the forefront of practical artificial intelligence (AI) deployment is the concept of model adaptation. In the context of LLMs, model adaptation involves modifying a pretrained model such as GPT-3.5 Turbo to enhance its performance on specific tasks or datasets. This process is important because while pretrained models offer a broad understanding of language and context, they may only excel in specialized tasks with adaptation.

Model adaptation encompasses a range of techniques, each designed to tailor a model’s vast general knowledge to particular applications. The path of model adaptation is not just about enhancing performance but about transforming a generalist AI model into a specialized tool adept at handling the nuanced demands of enterprise solutions.

9.1 What is model adaptation?

9.1.1 Basics of model adaptation

9.1.2 Advantages and challenges for enterprises

9.2 When to fine-tune an LLM

9.2.1 Key stages of fine-tuning an LLM

9.3 Fine-tuning OpenAI models

9.3.1 Preparing a dataset for fine-tuning

9.3.2 LLM evaluation

9.3.3 Fine-tuning

9.3.4 Fine-tuning training metrics

9.3.5 Fine-tuning using Azure OpenAI

9.4 Deployment of a fine-tuned model

9.4.1 Inference: Fine-tuned model

9.5 Training an LLM

9.5.1 Pretraining

9.5.2 Supervised fine-tuning

9.5.3 Reward modeling

9.5.4 Reinforcement learning

9.5.5 Direct policy optimization