Chapter 2: Chat: "Explain this code" is the worst prompt in AI
This chapter introduces a hands-on chat-based method for interrogating unfamiliar code with sharper follow-ups, bounded context, and prompts that produce useful explanations before scaling to workflows or agents.
After this chapter you will be able to
- Generic-answer prompt failure. Explain why pasting a file and asking "What does this do?" tends to produce generic text, and state that the problem is the vague question rather than necessarily the model.
- Chat scope and limits. Describe chat as a low-friction tool for exploring a few files at a time, and distinguish the cases where its limits mean a workflow or agent is needed.
- Generic answer as warm-up. Explain why the first response to a broad prompt such as “What does this do?” is useful even though it is generic. State that it provides vocabulary and a high-level map that help you formulate a sharper second question, rather than providing an actionable understanding by itself.
- Detail-driven follow-up questions. Describe how inspecting the code after the initial summary can reveal a surprising concrete detail worth asking about. Use the vocab_size discrepancy in nanoGPT as an example: GPT-2 has 50,257 tokens, while model.py uses 50,304 because it is rounded up to a multiple of 64 for GPU efficiency.
- Frontier model default. State the default model-selection rule: use the newest frontier model you can access. Explain that newer generations generally reason better, hallucinate less, and may also cost less, while unusually large codebases or deliberative design questions can require more than the default.
- Cross-file context requirements. Distinguish questions answerable from one file from questions that require multiple repository files. Explain why a file such as model.py can describe its own internals but cannot by itself support an end-to-end training or configuration explanation.
- Repository serialization crawler. Describe how the 20-line crawl.py script turns a repository into one pasteable text file. State that it walks the directory tree, retains source-like files, skips selected directories, and adds a banner identifying each file before joining the contents.
- Repository noise filtering. Explain why irrelevant repository files mislead an LLM and state the rule for filtering: keep files that describe what the system is, while cutting files that describe how it was built or tested.
- Concrete prompt constraints. Describe how to turn a vague code-explanation request into a concrete prompt by asking for a story rather than a category, numbers rather than adjectives, and a toy example rather than prose.
- Prompt-driven answer quality. Explain why a generic answer to 'explain this code' is primarily a prompt problem rather than an AI problem: without constraints, the model attempts to explain everything and therefore explains nothing in useful detail.
- Concrete user-story prompts. Describe how to replace a category question about a codebase's users or use cases with a prompt requesting one specific person and a walkthrough of that person's experience. Include the person's role, concrete goal, attempted alternative, exact frustration, and how the codebase resolves it.
- Numeric output constraints. Explain why concrete numerical limits work better than adjectives such as "simple," "concise," or "beginner-friendly" when prompting an LLM. State examples of usable constraints, such as a maximum number of nodes, words per node, or sentences per point.
- Memorable diagram boundaries. Describe how explicit diagram limits changed the forward-pass flowchart from an overwhelming, implementation-heavy result into a beginner-memorable architectural overview. Include the kinds of details that were excluded and the limits that shaped the improved diagram.
- Toy-example prompting. Describe how to turn an abstract explanation request into a request for a toy example with a minimal setup, specific numbers, and every intermediate calculation. Explain that the purpose is to make the mechanism visible rather than receiving more prose.
- Gradient accumulation equivalence. Explain, using the one-weight four-sample example, why two mini-batches of two samples can produce the same weight update as one batch of four when their mean gradients are scaled by gradient_accumulation_steps and accumulated before one update. State that gradient accumulation lets training use smaller batches to avoid holding a larger batch's activations in GPU memory at once.
- Active retrieval for retention. Explain why rereading is insufficient for retaining a codebase explanation and why active retrieval—writing, explaining, or testing yourself—resets the forgetting curve.
- Learning note refinement cycle. Describe the learning note as a personal Markdown study note and trace its repeated four-step cycle: skeleton from AI, follow-up questions, editing in your own words, and self-quiz.
- Disposable learning-note scaffold. Explain why the first AI-generated learning note should be a structured outline rather than polished prose, and why it is treated as disposable scaffolding to rewrite.
- Constrained learning-note prompt. Describe the prompt constraints for the learning note: simple editable markdown, aggressive brevity, an assumed knowledge level, and two bounded sections for concepts and important files.
- File dependency chain. Describe how the important-files section identifies what each selected file does and gives a dependency chain showing how files or generated artifacts lead to an outcome.
- Prioritization through numeric limits. Justify why limits such as eight concepts and eight files improve the first note: they force prioritization, keep the result screen-sized, and leave missing topics for follow-up questions instead of producing long jargon lists.
- Concrete jargon follow-ups. Describe how to follow up on a line in your notes that you think you understand by asking for a concrete example and a distinction from a related term. Explain that recognizing terminology is not enough; you should be able to define it in your own words.
- Self-attention and cross-attention. Distinguish self-attention from cross-attention using the section's sentence example. State that self-attention connects tokens within one sequence, whereas cross-attention lets tokens in one sequence attend to tokens in a different sequence.
- Own-word rephrasing. Explain why rewording an AI answer in your own words is the essential learning step, rather than copying its output. Describe how a stalled rewrite identifies the exact point you do not yet understand.
- Retrieval testing over re-reading. Explain why re-reading notes can create an illusion of competence and why testing yourself is the way to find out what you actually learned.
- Code-specific quiz prompts. Describe how to ask the AI for a difficult multiple-choice quiz after studying code: request a small number of questions that connect specific code parts and use real file names and variable names.
- Missed-question learning loop. Trace the follow-up cycle after a quiz: add missed questions to your notes, learn the correct answers, focus on those gaps, and repeat the test.
- Chat scalability limits. Explain why chat-based exploration works for a small set of files but does not scale to a large codebase. Describe that it requires the developer to paste files, ask questions, follow up, and identify which details are worth investigating.
- Workflow and agent roles. Distinguish workflows from agents as the two tools introduced after chat: workflows automate repetitive tasks to scale beyond small repositories, while agents explore independently when you do not know what to ask upfront.
- Workflow loop over files. Describe a workflow as a script that calls an LLM in a loop across every file in a repository. Explain that this converts repeated manual tasks—asking what files do, how they connect, and writing learning notes—into a pipeline applicable to any repository.
- Per-file architecture limitation. Distinguish useful per-file analysis from architecture understanding: per-file LLM summaries work well for individual files but miss connections between files, where the actual architecture lives. State that the next chapter addresses capturing those connections.
- Agent autonomy. Explain that an agent is an LLM given a goal that decides its own next steps, rather than requiring the user to specify every file or action.
- Read write run cycle. Describe how an agent can read related files, write artifacts, and run commands while pursuing a codebase question, using the read, write, and run cycle to operate autonomously.
- Concentration coverage tradeoff. Distinguish chat's token-efficient deep focus on a few supplied files from an agent's broader multi-file coverage, whose context is consumed by files, tool outputs, and reasoning steps.
- Complementary tool limitations. Explain how chat, workflows, and agents each have a distinct weakness that another tool can offset: chat has limited scale, workflows cannot adapt, and agents can wander without focus.
- Workflow-to-agent feedback loop. Trace the feedback loop in which a repeatable chat-based exploration process becomes a workflow, whose structured summary is loaded as agent context so the agent can navigate directly to relevant files.
What it builds on
- Three questions you can run today (chapter 1) · It assumes that asking AI targeted questions can be an engineering practice.
- The reframe: From reading code to architectural thinking (chapter 1) · The shift from merely reading code toward reasoning about the architectural and design choices behind it.
Section by section
Introduction
This chapter introduces a hands-on chat-based method for interrogating unfamiliar code with sharper follow-ups, bounded context, and prompts that produce useful explanations before scaling to workflows or agents.
- Chapter learning path: Describe the chapter's planned progression from trying a poor "What does this do?" prompt to using sharper follow-ups and producing a one-page codebase overview.
Why it matters: Knowing the path helps you treat chat as an exercise for generating design insights rather than as a passive code explainer. - Generic-answer prompt failure: Explain why pasting a file and asking "What does this do?" tends to produce generic text, and state that the problem is the vague question rather than necessarily the model.
Why it matters: This reframe prevents you from dismissing AI when an unbounded prompt produces an answer that could fit almost any code. - Chat scope and limits: Describe chat as a low-friction tool for exploring a few files at a time, and distinguish the cases where its limits mean a workflow or agent is needed.
Why it matters: Choosing chat for a bounded investigation and other tools for larger or self-directed investigations keeps your code-understanding process effective. - nanoGPT practice codebase: State why nanoGPT is used for the chapter exercise and identify the relevant scope: its 19-file repository, the 330-line model.py core architecture file, and its role as unfamiliar code to interrogate.
Why it matters: The deliberately small but unfamiliar example lets you practice the method without requiring prior machine-learning experience.
Key terms: sharper follow-ups, one-page overview, design insight, "What does this do?", generic text, effective follow-up question, chat, workflow, agent, context window
2.1 The first answer is always generic. That's the point.
This section shows that a generic first code explanation is a useful warm-up for finding specific follow-up questions, not the final understanding.
- Generic answer as warm-up: Explain why the first response to a broad prompt such as “What does this do?” is useful even though it is generic. State that it provides vocabulary and a high-level map that help you formulate a sharper second question, rather than providing an actionable understanding by itself.
Why it matters: Treating the first answer as a starting point helps you turn a vague code-reading session into focused investigation. - Detail-driven follow-up questions: Describe how inspecting the code after the initial summary can reveal a surprising concrete detail worth asking about. Use the vocab_size discrepancy in nanoGPT as an example: GPT-2 has 50,257 tokens, while model.py uses 50,304 because it is rounded up to a multiple of 64 for GPU efficiency.
Why it matters: A specific anomaly can expose the design decision that a file-level summary hides.
Key terms: generic summary, warm-up, vocabulary, sharper second question, vocab_size, 50,304, 50,257, multiple of 64, GPU efficiency, tensor cores
2.1.1 Which model should I pick?
This section gives a practical default for choosing an AI model and explains when privacy requirements justify running a local model instead.
- Frontier model default: State the default model-selection rule: use the newest frontier model you can access. Explain that newer generations generally reason better, hallucinate less, and may also cost less, while unusually large codebases or deliberative design questions can require more than the default.
Why it matters: A simple default prevents model selection from becoming a distraction before you start understanding the code. - Local model privacy tradeoff: Describe when to run an open-source model locally: when company policy forbids sending code to a cloud API. State that tools such as Ollama or LM Studio can run models such as Llama, Qwen, or DeepSeek on the machine so code stays in the environment, with weaker complex-architecture answers as the tradeoff.
Why it matters: This gives you a usable AI option when code cannot leave your company's environment.
Key terms: newest frontier model, reasoning, hallucinates, codebase too large to paste, design question, deliberation, open-source model, locally, cloud API, Llama
2.2 One file can't answer cross-file questions. 20 lines fix that.
This section shows how serializing the relevant repository into one pasteable text file lets chat answer cross-file questions, within context-window limits.
- Cross-file context requirements: Distinguish questions answerable from one file from questions that require multiple repository files. Explain why a file such as model.py can describe its own internals but cannot by itself support an end-to-end training or configuration explanation.
Why it matters: The AI can only reason over the code you provide, so the scope of your paste must match the scope of your question. - Repository serialization crawler: Describe how the 20-line crawl.py script turns a repository into one pasteable text file. State that it walks the directory tree, retains source-like files, skips selected directories, and adds a banner identifying each file before joining the contents.
Why it matters: Serialization gives a chat interface, which accepts text rather than directories, the multi-file context needed for repository-level questions. - Context-window tradeoffs: Explain why pasting an entire repository is limited by the model's context window. Describe how the code, the question, and space for the answer share that window, and why irrelevant code can degrade answers even before the hard limit is reached.
Why it matters: Knowing the context tradeoff prevents a whole-repository paste from becoming unusable on a larger codebase.
Key terms: single-file view, cross-file questions, whole-repo dump, end-to-end pipeline, serialize, crawl.py, pasteable file, EXTS, SKIP_DIRS, directory tree
2.2.1 What you cut matters as much as what you keep
This subsection explains how mechanically filtering repository noise gives an LLM a more accurate architectural view of a codebase.
- Repository noise filtering: Explain why irrelevant repository files mislead an LLM and state the rule for filtering: keep files that describe what the system is, while cutting files that describe how it was built or tested.
Why it matters: Reducing noise keeps the model focused on your application’s architecture instead of spending its answer on incidental tooling. - Common file exclusion categories: Describe the usual categories to cut from a repository dump: tests, dependencies, build output, lock files, and binary media or data, including why each does not serve as the application’s architectural source of truth.
Why it matters: Recognizing these recurring noise categories lets you prepare useful code context across unfamiliar ecosystems. - Filtering defaults and scale: State how the default SKIP_DIRS and EXTS lists in crawl.py handle common noise, and explain when filtering becomes increasingly important as repository size grows while inclusion remains safer when uncertain.
Why it matters: You can rely on sensible defaults for ordinary repositories while knowing when larger repositories require more deliberate selection.
Key terms: mechanically filter files, architecture, source of truth, SKIP_DIRS, EXTS, tests, dependencies, node_modules, build output, dist/
2.3 A concrete prompt beats a vague one, every time
This section shows that concrete prompt constraints produce useful codebase explanations by replacing vague requests with stories, numbers, and checkable examples.
- Concrete prompt constraints: Describe how to turn a vague code-explanation request into a concrete prompt by asking for a story rather than a category, numbers rather than adjectives, and a toy example rather than prose.
Why it matters: Concrete constraints give you an answer you can inspect instead of an impressive-sounding summary you cannot use. - Prompt-driven answer quality: Explain why a generic answer to 'explain this code' is primarily a prompt problem rather than an AI problem: without constraints, the model attempts to explain everything and therefore explains nothing in useful detail.
Why it matters: This keeps you from blaming the model when the practical fix is to ask a question with a concrete target. - Checkable output forms: Explain why named people and stories, numbers, diagrams or tables, and worked toy examples make AI output more useful: they are constraints that are harder to fake and easier for a developer to trace or check.
Why it matters: Outputs you can trace and verify help you build understanding instead of merely recognizing polished language.
Key terms: concrete prompt, story, category, numbers, adjectives, toy example, generic, abstract summary, constraints, concrete answer
2.3.1 Ask for a story, not a category
This section shows that asking AI for a concrete user story with a specific goal and frustration produces a more memorable understanding of a codebase than asking for audience or use-case categories.
- Concrete user-story prompts: Describe how to replace a category question about a codebase's users or use cases with a prompt requesting one specific person and a walkthrough of that person's experience. Include the person's role, concrete goal, attempted alternative, exact frustration, and how the codebase resolves it.
Why it matters: A concrete story gives you a usable picture of why someone would choose the codebase instead of a generic audience list. - Story-based memory advantage: Explain why a concrete character facing a goal and obstacle is easier to remember than an abstract list of user categories. Relate the advantage to the brain compressing structured stories into memorable information.
Why it matters: If you remember the user's concrete problem, you are more likely to remember what the codebase is for and why its design matters. - nanoGPT versus HuggingFace scenario: Trace the example contrast in which a researcher needs access to raw attention weights, gets blocked by layered HuggingFace training behavior and hidden output issues, and can inspect and modify nanoGPT's small model implementation directly. State that the example illustrates a specific frustration and resolution rather than a general claim that one tool is always better.
Why it matters: The contrast demonstrates the level of concrete detail that makes a user story explain a project's appeal.
Key terms: category questions, specific person, specific frustration, walk me through their experience, character, goal, obstacle, structure sticks, compresses information, nanoGPT
2.3.2 Constrain by numbers, not adjectives
This section shows that replacing vague descriptive requests with measurable numeric constraints produces clearer, more memorable AI output.
- Numeric output constraints: Explain why concrete numerical limits work better than adjectives such as "simple," "concise," or "beginner-friendly" when prompting an LLM. State examples of usable constraints, such as a maximum number of nodes, words per node, or sentences per point.
Why it matters: Measurable limits give the model boundaries it can follow instead of leaving it to improvise what "simple" means. - Memorable diagram boundaries: Describe how explicit diagram limits changed the forward-pass flowchart from an overwhelming, implementation-heavy result into a beginner-memorable architectural overview. Include the kinds of details that were excluded and the limits that shaped the improved diagram.
Why it matters: A diagram is useful for understanding architecture only when its shape prevents distracting detail from crowding out the main flow. - Quantified substance requests: State that numerical constraints can govern an explanation's substance as well as a diagram's layout: when the model hand-waves, ask it for numbers to compute.
Why it matters: Requesting computations can turn an abstract or hand-wavy explanation into something concrete enough to inspect.
Key terms: output constraints, adjectives, numbers, concrete constraints, Mermaid flowchart, forward pass, nodes, arrow labels, implementation details, architecture
2.3.3 Ask for toy examples, not prose
This section shows that requesting a small numerical toy example makes an abstract code concept concrete enough to explain and verify.
- Toy-example prompting: Describe how to turn an abstract explanation request into a request for a toy example with a minimal setup, specific numbers, and every intermediate calculation. Explain that the purpose is to make the mechanism visible rather than receiving more prose.
Why it matters: A numerical walkthrough gives you an explanation you can check, repeat, and teach to someone else. - Gradient accumulation equivalence: Explain, using the one-weight four-sample example, why two mini-batches of two samples can produce the same weight update as one batch of four when their mean gradients are scaled by gradient_accumulation_steps and accumulated before one update. State that gradient accumulation lets training use smaller batches to avoid holding a larger batch's activations in GPU memory at once.
Why it matters: This example demonstrates how a numerical walkthrough can prove that an implementation detail has the intended effect. - Numerical accumulation walkthrough: Trace the section's toy calculation: with w = 0, the four per-sample gradients are -2, -8, -18, and -32; their full-batch mean is -15, while mini-batch means of -5 and -25 become -2.5 and -12.5 after division by two and sum to -15. State that a 0.1 learning rate changes the weight to 1.5 in either case.
Why it matters: Walking through the numbers is what turns the claim about equivalent updates into something you can independently explain on a whiteboard.
Key terms: toy example, real numbers, abstract explanation, gradient_accumulation_steps, gradient accumulation, mini-batch, gradient, forward pass, backward pass, w
2.4 You'll forget two-thirds by tomorrow
This section explains why passive rereading quickly fades and introduces a repeatable AI-assisted learning-note cycle for retaining codebase understanding.
- Active retrieval for retention: Explain why rereading is insufficient for retaining a codebase explanation and why active retrieval—writing, explaining, or testing yourself—resets the forgetting curve.
Why it matters: Retrieval turns a temporary AI explanation into understanding you can still use tomorrow. - Learning note refinement cycle: Describe the learning note as a personal Markdown study note and trace its repeated four-step cycle: skeleton from AI, follow-up questions, editing in your own words, and self-quiz.
Why it matters: Repeating this cycle makes the note reflect your own grasp instead of leaving you with an AI-generated answer you only recognize.
Key terms: forgetting curve, active retrieval, comprehension debt, learning note, Skeleton, Follow-up, Edit, Quiz, CODEBASE.md
2.4.1 Let AI write the first draft. It's disposable.
Use AI to generate a deliberately short, editable learning-note outline that prioritizes core concepts, important files, and their dependency chain.
- Disposable learning-note scaffold: Explain why the first AI-generated learning note should be a structured outline rather than polished prose, and why it is treated as disposable scaffolding to rewrite.
Why it matters: A disposable first draft gets you to a useful map quickly without making you preserve weak wording or incomplete understanding. - Constrained learning-note prompt: Describe the prompt constraints for the learning note: simple editable markdown, aggressive brevity, an assumed knowledge level, and two bounded sections for concepts and important files.
Why it matters: Specific limits force the model to select what matters instead of producing a broad but unusable explanation. - Code-grounded concept list: Explain that each concept in the first section should be in plain English and point to where it appears in the code, such as a file, function, class, or identifier.
Why it matters: Mapping concepts to code turns abstract jargon into places you can inspect and question next. - File dependency chain: Describe how the important-files section identifies what each selected file does and gives a dependency chain showing how files or generated artifacts lead to an outcome.
Why it matters: A dependency chain gives a navigable story of how the codebase's pieces connect instead of a disconnected file list. - Prioritization through numeric limits: Justify why limits such as eight concepts and eight files improve the first note: they force prioritization, keep the result screen-sized, and leave missing topics for follow-up questions instead of producing long jargon lists.
Why it matters: The limits create a useful starting map that you can interrogate, rather than overwhelming you with definitions that do not teach the codebase.
Key terms: structured outline, polished draft, disposable scaffolding, learning note, simple markdown, aggressively short, Concepts I need to know, The important files, max 8 items, plain English
2.4.2 Follow up on the line you think you understand
This section shows how to turn a familiar-sounding jargon term into genuine understanding by asking a concrete follow-up that distinguishes it from a related concept.
- Concrete jargon follow-ups: Describe how to follow up on a line in your notes that you think you understand by asking for a concrete example and a distinction from a related term. Explain that recognizing terminology is not enough; you should be able to define it in your own words.
Why it matters: A targeted follow-up exposes the gaps hidden behind a sentence that merely sounds familiar. - Self-attention and cross-attention: Distinguish self-attention from cross-attention using the section's sentence example. State that self-attention connects tokens within one sequence, whereas cross-attention lets tokens in one sequence attend to tokens in a different sequence.
Why it matters: This distinction demonstrates the level of explanation needed to move from recognizing jargon to actually understanding it. - Decoder-only attention choice: Explain why nanoGPT, as a decoder-only GPT model, uses self-attention rather than cross-attention. State that it processes one sequence to predict the next token, so it has no second sequence for cross-attention; cross-attention is associated with encoder-decoder input-to-output tasks such as translation.
Why it matters: Connecting an architectural label to the attention mechanism prevents the terms from becoming isolated definitions.
Key terms: concepts table, jargon term, concrete example, self-attention, cross-attention, sequence, encoder-decoder, nanoGPT, decoder-only, next token
2.4.3 The step everyone skips is the only one that works
This section argues that rewriting an AI explanation in your own words exposes gaps in understanding and frees working memory for the next concept.
- Own-word rephrasing: Explain why rewording an AI answer in your own words is the essential learning step, rather than copying its output. Describe how a stalled rewrite identifies the exact point you do not yet understand.
Why it matters: Putting an explanation into your own words turns apparent recognition into a test of real understanding. - Mechanisms over metaphors: Distinguish a metaphorical explanation from a mechanism-level rewording using the self-attention example. State that saying tokens look at earlier tokens is a metaphor, whereas a mechanism-level account says each token scores earlier tokens and forms a weighted average using attention weights.
Why it matters: Replacing a memorable metaphor with the underlying steps checks whether you can explain what the code or model is actually doing. - Written memory offloading: Describe how writing a concept down offloads it from working memory, allowing attention to move to the next concept. Use the captured distinction between self-attention and cross-attention as the section's example.
Why it matters: Capturing an idea on the page means you no longer need to keep it mentally active while learning what comes next.
Key terms: rewording, working memory, self-attention, token, weighted average, attention weights, cross-attention
2.4.4 Re-reading feels like learning. It isn't.
This section replaces passive re-reading with difficult, code-specific self-quizzing and uses missed answers to drive the next learning cycle.
- Retrieval testing over re-reading: Explain why re-reading notes can create an illusion of competence and why testing yourself is the way to find out what you actually learned.
Why it matters: Recognition feels familiar, but recalling and applying the material exposes whether you can really use it. - Code-specific quiz prompts: Describe how to ask the AI for a difficult multiple-choice quiz after studying code: request a small number of questions that connect specific code parts and use real file names and variable names.
Why it matters: Questions tied to real relationships in the repository test whether you understand behavior across the code rather than merely remember isolated terms. - Missed-question learning loop: Trace the follow-up cycle after a quiz: add missed questions to your notes, learn the correct answers, focus on those gaps, and repeat the test.
Why it matters: Wrong answers identify the most valuable next targets for study instead of hiding weak spots behind a polished summary. - Companion repository resources: State that the companion repository provides ready-to-paste templates for the chapter's techniques, a learning-note skeleton, and a worked learning note whose missed quiz questions and corrected answers are especially useful.
Why it matters: The provided materials let you apply the process immediately and show that documented mistakes are part of useful learning evidence.
Key terms: illusion of competence, re-reading, test yourself, quiz questions, real file names, real variable names, multiple choice, missed questions, gaps in your knowledge, learning note
2.5 Chat can't scale past five files. That's not a bug.
This section explains why manual chat-based code exploration stops scaling and positions workflows and agents as progressively more autonomous alternatives.
- Chat scalability limits: Explain why chat-based exploration works for a small set of files but does not scale to a large codebase. Describe that it requires the developer to paste files, ask questions, follow up, and identify which details are worth investigating.
Why it matters: Recognizing chat's limits helps you choose a tool that can handle a repository without relying on your prior knowledge of it. - Workflow and agent roles: Distinguish workflows from agents as the two tools introduced after chat: workflows automate repetitive tasks to scale beyond small repositories, while agents explore independently when you do not know what to ask upfront.
Why it matters: The distinction lets you select automation for a known repeatable process or autonomy for an unfamiliar codebase. - Tool autonomy spectrum: Describe the single axis used to compare chat, workflows, and agents: how much work the developer performs versus how much the tool performs. State that chat requires the developer to drive every step, a workflow runs the developer's script, and an agent chooses its own moves.
Why it matters: This spectrum provides a simple way to reason about the tradeoff among the three ways of working with a codebase.
Key terms: chat, codebase, follow-up questions, workflows, agents, repetitive tasks, explore the codebase independently, workflow, agent, drive every step
2.5.1 What's a workflow?
This section defines an LLM workflow as reusable code that calls a model across repository files, turning manual chat-based investigation into a controllable pipeline.
- Programmatic LLM access: Explain that major LLM vendors provide APIs that let a Python program or another language send prompts and code files to the same models available in a chat window. State that the essential difference is that the question lives in a script that can be run again.
Why it matters: API access is what makes repeatable, automated codebase analysis possible instead of one-off chat conversations. - Workflow loop over files: Describe a workflow as a script that calls an LLM in a loop across every file in a repository. Explain that this converts repeated manual tasks—asking what files do, how they connect, and writing learning notes—into a pipeline applicable to any repository.
Why it matters: A workflow lets you systematically investigate a codebase at a scale that manual file-by-file chat cannot sustain. - Workflow input and prompt control: Explain that a workflow gives the developer control over what the model sees, the order in which it sees it, and the questions it receives. Relate this control to directing the same pipeline at different repositories.
Why it matters: Controlling context, sequence, and prompts lets you make automated analysis deliberate rather than treating the model as an opaque black box. - Per-file architecture limitation: Distinguish useful per-file analysis from architecture understanding: per-file LLM summaries work well for individual files but miss connections between files, where the actual architecture lives. State that the next chapter addresses capturing those connections.
Why it matters: Recognizing this boundary prevents you from mistaking a collection of file summaries for an understanding of the system's architecture.
Key terms: programmatic API access, LLM API, chat window, script, workflow, for loop, pipeline, per-file approach, repository, what the model sees
2.5.2 What's an agent?
This section defines coding agents, shows how they autonomously inspect and act on a codebase, and explains their coverage-versus-concentration tradeoff with chat.
- Agent autonomy: Explain that an agent is an LLM given a goal that decides its own next steps, rather than requiring the user to specify every file or action.
Why it matters: This helps you choose when to delegate exploration to an agent instead of manually directing a chat prompt. - Read write run cycle: Describe how an agent can read related files, write artifacts, and run commands while pursuing a codebase question, using the read, write, and run cycle to operate autonomously.
Why it matters: The agent is useful because it can gather evidence and leave behind useful output, not merely generate an explanation. - Concentration coverage tradeoff: Distinguish chat's token-efficient deep focus on a few supplied files from an agent's broader multi-file coverage, whose context is consumed by files, tool outputs, and reasoning steps.
Why it matters: Knowing this tradeoff lets you use chat for close reading and agents for wider exploration instead of expecting either mode to excel at both.
Key terms: agent, LLM, goal, autonomously, read, write, and run cycle, CODEBASE.md, get_batch(), np.memmap, concentration, coverage
2.5.3 Chained together, each tool covers the next one's weakness
This section explains how chat, workflows, and agents compensate for one another’s limits when chained into a codebase-understanding loop.
- Complementary tool limitations: Explain how chat, workflows, and agents each have a distinct weakness that another tool can offset: chat has limited scale, workflows cannot adapt, and agents can wander without focus.
Why it matters: Choosing tools as complements lets you avoid treating any one AI interface as a complete solution. - Exploration and editing tool choice: Describe when to start with chat or an agent for open-ended codebase exploration, and distinguish the agent's editing abilities from chat's text-only output.
Why it matters: Starting with the convenient exploration tool and using an agent for code changes keeps investigation practical. - Workflow-to-agent feedback loop: Trace the feedback loop in which a repeatable chat-based exploration process becomes a workflow, whose structured summary is loaded as agent context so the agent can navigate directly to relevant files.
Why it matters: This loop prevents repeated manual investigation and reduces an agent's wasted effort rediscovering the codebase.
Key terms: chat, workflow, agent, chaining, structured output, exploration, open-ended question, conversational thread, repeatable process, consistent pipeline
2.6 Summary
This summary recaps a disciplined chat-based learning loop—careful context, precise prompts, follow-ups, and notes—and positions chat as preparation for scalable workflows and agents.
Where this is used later
- Turn any codebase into a tutorial for $2 (chapter 3) · The workflow chapter addresses codebases too large or complex for manual chat context.
- Broad questions: Your agent skimmed 25 files and missed everything (chapter 4) · The agent chapter addresses investigations where the AI must decide what files to inspect.
- Prune the obvious, then let the LLM pick the rest (chapter 3) · The use of an LLM call to obtain a model response.
- What the LLM sees determines everything (chapter 3) · Choosing what the LLM sees becomes the central problem when a whole repository cannot simply be pasted.
- Most repos fit. The interesting question is what to do when they don't. (chapter 3) · The discussion of repositories that do not fit builds directly on the context-window constraint.
- The LLM needs guidance to pick well (chapter 3) · Guiding the LLM’s file selection depends on understanding which files are architectural signal and which are noise.
- Summary (chapter 3) · the importance of deciding what to keep and cut from supplied context
- Every codebase builds for someone, even a library (chapter 5) · Reverse-engineering product intent requires reasoning about the particular people and needs a codebase serves.
Concepts explained in this chapter
- pipelineA workflow is a repeatable sequence of predefined steps that systematically transforms inputs into consistent outputs.
- learning noteA learning note is concise instructional documentation that records the concepts, files, and relationships needed to understand a codebase.
- context windowA context window is the finite token capacity available to a language model for processing input, tool results, and generated reasoning or output.
- repositoryA repository is a structured collection of a software project's files and directories managed as a unit.
- nanoGPTnanoGPT is a compact GPT-style implementation for training and generating text with a decoder-only transformer.
- promptA prompt is an instruction or input supplied to an AI model to elicit a particular response.
- AI agentAn AI agent is a software system that uses artificial intelligence to pursue tasks through autonomous or semi-autonomous actions.
- language modelA large language model is a machine-learning model trained on large amounts of text to generate and transform language from prompts.
- contractAn external API is an interface exposed by a separately operated system for client software to access its operations or data.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
