chapter nine

9 Tailoring models with model adaptation and fine-tuning

 

This chapter covers

  • Basics of Model adaptation and their advantages
  • Understanding how to train an LLM
  • Learning how to fine-tune an LLM using both SDKs and the GUI
  • Best practices for evaluation criteria and metrics for fine-tuned LLMs
  • Deploying a fine-tuned model for inference
  • Gaining insight into key model adaptation techniques – SFT, PEFT, LoRA, and RLHF

As we delve into the intricate world of LLMs, a key aspect that stands at the forefront of practical AI deployment is the concept of Model Adaptation. Model adaptation in the context of LLMs involves modifying a pre-trained model such as GPT3.5 Turbo to enhance its performance on specific tasks or datasets. This process can be important because while pre-trained models offer a broad understanding of language and context, they may only excel in specialized tasks with adaptation.

Model adaptation encompasses a range of techniques, each designed to tailor a model's vast general knowledge to particular applications. The path of model adaptation is not just about enhancing performance; it's about transforming a generalist AI model into a specialized tool adept at handling the nuanced demands of enterprise solutions.

9.1 What is model adaptation

9.1.1 Basics of model adaptation

9.1.2 Advantages and Challenges for Enterprises

9.2 When to fine-tune an LLM?

9.2.1 Key stages for fine-tuning an LLM

9.3 Fine-tuning OpenAI Models

9.3.1 Preparing dataset for fine-tuning

9.3.2 LLM Evaluation

9.3.3 Fine-tuning

9.3.4 Fine-Tuning Training Metrics

9.3.5 Fine-tuning using Azure OpenAI

9.4 Deployment of a fine-tuned Model

9.4.1 Inference – Fine-tuned model

9.5 Training an LLM

9.6 Model Adaptation Techniques

9.6.1 Low-Rank Adaptation (LoRA)

9.7 RLHF Overview

9.7.1 Challenges with RLHF

9.7.2 Scaling an RLHF implementation

9.8 Summary

9.9 References