chapter two

2 LLM Basics

 

This chapter covers

  • How LLMs process text through tokens, embeddings, and attention
  • How a raw language model becomes a conversational assistant through pretraining, fine-tuning, and alignment
  • What guardrails are and how they emerge from training
  • Why context windows define the boundaries of what a model can see

You don't need to understand how an engine works to drive a car. But if you've ever driven on ice, you know that understanding what happens when you brake changes how you react. A general working knowledge of the tool you are using means that you can utilize it more efficiently and safely, and solve problems when they arise.

LLMs work the same way. You can use them without knowing anything about tokens, embeddings, or context windows. Most people do. But the moment something goes wrong, such as a response that loses the thread, an answer that sounds right but isn't, or a model that ignores part of your instructions, you're left guessing. Was it the phrasing? The length? Something else entirely?

This chapter answers those questions, not by going deep into the mathematics, but by showing you enough of what happens under the surface that the behavior starts to make sense. Once it makes sense, you can stop guessing, and start steering.

2.1 How LLMs Work Internally

2.1.1 From Text to Tokens

2.1.2 From Tokens to Meaning: Embeddings

2.1.3 From Meaning to Context

2.1.4 Hitting the Limit: The Context Window

2.2 How LLMs Are Trained As Assistants

2.2.1 Pretraining: Learning the Patterns of Language

2.2.2 Fine-Tuning: Teaching the Model to Follow Instructions

2.2.3 Human Feedback and Alignment

2.2.4 Specialized Models

2.3 Summary

2.4 References