A large language model (LLM) takes text and generates text. In a direct chat, you enter a prompt and receive a response. The model can also sit inside an assistant or agent, where its output helps guide tool use and other work. section 2.1.1 section 1.1.1
Think of generation as choosing one token at a time. Tokenization first converts text into numbered tokens. Given an input, the model assigns probabilities to possible next tokens; a separate sampling step chooses one. The decoding strategy can change that choice even when the model produces the same probability distribution for a given input. section 2.1.1
Figure 2.1 The training cycle and inference/generation process of a large language model. On the left is the model’s training process, in which documents are ingested to train the LLM in a first pass. On the right, a user enters a prompt, which is first tokenized and fed into the model. The model then outputs probabilities it uses to sample and produce the next token.
Follow the two paths: training uses documents to improve next-token predictions, while generation starts with a tokenized user prompt and samples from predicted probabilities. section 2.1.1
During training, prediction errors are used to adjust the model’s weights over many iterations. A later alignment phase, such as reinforcement learning with human feedback, helps it follow instructions and avoid harmful output; that phase is distinct from teaching new knowledge or improving reasoning. section 2.1.1
Table 1.1 The four LLM interaction patterns covered in this book
| Pattern |
Approval model |
Autonomy level |
Typical use case |
Example platform |
|---|---|---|---|---|
| Direct LLM chat |
No tools invoked; the model only generates text |
None |
Q&A, drafting, brainstorming, learning |
Original ChatGPT (2022), early Claude (2023) |
| Tool-augmented LLM |
Implicit per call; a user prompt triggers a single tool |
Low |
Image generation, web search, simple lookups |
ChatGPT with DALL·E, Gemini with Google Search |
| Assistant |
Per-task approval; the user confirms each action |
Medium |
Pair programming, cited research, document editing |
GitHub Copilot Chat, Cursor, Claude in Excel |
| Agent |
Goal-level approval with gated high-stakes actions; the user sets the objective, and the agent decides the steps |
High |
Multistep research, repo-wide refactors, browser automation |
Claude Code, OpenAI Operator, Devin |
Compare four ways to use the same core capability: direct text generation, a tool-augmented call, an assistant that seeks task approval, and an agent that chooses steps toward a goal with high-stakes actions gated. section 1.1.1
Make a request specific
Instead of asking only for a time-travel story, specify that it takes the traveler to 1921, stays within five paragraphs, and uses a humorous first-person voice. For a tool-using search task, instructions can also state which sources to choose, how long to search, and how to format the result. section 2.2.2
For practical control, measure tokens rather than assuming text length reflects cost or context use. Adjust generation settings for the desired balance of creativity, consistency, and expense; use clear prompts and typed output schemas when a larger workflow must handle the model’s variable output. Tools, memory, and orchestration—not the model alone—turn its reasoning into actions. section 2.6 section 5.2
