Part 1. Overview

This part of the book presents the fundamentals of what RLHF is and what it does to models, while offering general context for what will follow. It also covers major breakthroughs leading up to ChatGPT and discusses how reinforcement learning from human feedback (RLHF) compares to classical reinforcement learning for control problems.

Chapter 1 is an introduction to the topic, chapter 2 provides a quick recap of the core research that led to the breakthrough that was ChatGPT’s RLHF, and chapter 3 recaps the fundamentals of RL while foreshadowing the key tools detailed in this book.

When you’re finished with this part, you’ll be ready to learn about the complex, technical algorithms and data that will let you successfully post-train a modern language model.