Chapter 1: Introduction to prompt programming and DSPy

Introduction

This introduction explains why manual prompt tuning is difficult and how DSPy replaces it with a code-based, evaluable approach to prompt programming.

• Prompt sensitivity and trial and error: Explain why finding a reliable prompt can be difficult even when an LM produces an immediate response.
  Why it matters: This is the problem that motivates a more systematic way to develop LM applications.
• Code-based prompt programming: Distinguish DSPy's prompt programming approach from manually rephrasing prompts in prompt engineering.
  Why it matters: It shifts developers' attention from exact wording to the task they want the LM to perform.
• Prompt evaluation and optimization: Describe what optimizing prompts means here and why DSPy's support for it is useful.
  Why it matters: Evaluation gives prompt selection a basis beyond repeated guesses about wording.

Key terms: prompts, prompt engineering, trial-and-error, prompt programming, DSPy, evaluating, optimizing, candidate prompts

1.1 What makes prompt programming different?

This section frames prompt programming as a higher-level evolution of manually developing prompts.

• Higher-level prompt programming: Explain why the section presents prompt programming as a natural evolution from manually developing prompts, using its comparison with other higher-level approaches in computing.
  Why it matters: This gives you the section’s basic reason for treating prompt programming as a shift in how prompts are developed.

Key terms: prompt programming, manually developing prompts, higher-level concepts, lower-level details

1.1.1 The goals of prompt programming

This section presents prompt programming as a higher-level way to specify an LM application and lists the goals of tools such as DSPy.

• Higher-level application code: Explain how prompt programming lets developers specify what an LM-based application will do without directly constructing prompts or calling the LM API for most tasks.
  Why it matters: This is the central shift in how the section proposes building LM applications.
• Quality and development workflow goals: State that prompt programming tools aim to improve prompt quality, reduce development time, and make prompt development code easy to re-execute.
  Why it matters: These goals explain why the higher-level approach may be useful in a development workflow.
• Model and technique flexibility: State that prompt programming tools aim to make switching between LMs and experimenting with prompting techniques easier.
  Why it matters: These goals connect higher-level code to the ability to try alternatives.

Key terms: prompt programming, top layer, LM’s API, higher-level code, prompt quality, development time, re-execution of prompt development code, LMs, prompting techniques

1.1.2 Prompt programming vs. prompt engineering

This section contrasts the difficulties of manually engineering prompts with DSPy’s methodical approach to generating, evaluating, and selecting prompts.

• Limits of manual prompting: Explain why manually creating and tuning prompts can become difficult when prompt quality matters, including the effort of testing changes and repeating work for different LMs.
  Why it matters: These difficulties explain what prompt programming is intended to improve.
• DSPy prompt optimization: Describe how DSPy takes general task instructions, generates and evaluates candidate prompts, refines them based on performance, and selects a best-performing prompt.
  Why it matters: It gives a high-level picture of how DSPy replaces much manual trial and error with evaluated prompt candidates.

Key terms: prompt engineering, manually creating and tuning prompts, LMs, DSPy, prompt programming, prompt optimization, candidate prompts

1.2 Introducing DSPy

This section explains what DSPy stands for and how its name reflects its approach to language models.

• Meaning of the DSPy name: Explain that DSPy means Declarative Self-improving Python: users declare what they want an LM to do rather than write a specific prompt, DSPy can automatically improve the prompts it generates, and it is developed in Python.
  Why it matters: The name captures the basic shift from writing prompts yourself to specifying a task and allowing prompt optimization.

Key terms: DSPy, Declarative Self-improving Python, declarative, self-improving, optimization process

1.2.1 DSPy code example

This section uses a short question-answering example to show how DSPy organizes LM work as clear, interchangeable code rather than handwritten prompts.

• Question answering code flow: Describe the basic sequence in the example: specify an LM, define question answering as the task, query the LM, and collect its response. Note that this example contains no written prompt or prompt optimization.
  Why it matters: It gives you a concrete picture of what a simple DSPy application does.
• Clear functions for development: Explain how functions with clear inputs, outputs, and purposes make DSPy code precise and easier to test, debug, review, and monitor than working directly with prompts.
  Why it matters: Clear code makes an LM feature easier to maintain and check.
• Interchangeable models and techniques: Explain how DSPy's modular organization lets developers replace the LM or prompting technique without changing the rest of the code, while a common LM-call API avoids learning a separate call signature for each LM.
  Why it matters: You can try alternatives without rewriting the application around each choice.

Key terms: LM, question answering, prompts, inputs and outputs, testing, debugging, code reviews, monitoring in production, modularity, Predict module

1.2.2 Advantages of working with code

This section explains why treating prompts as source code can make LLM application development more disciplined and reliable.

• Code Based Prompt Development: Explain how prompt programming applies established software-development practices to avoid common weaknesses of ad hoc prompt engineering.
  Why it matters: A code-based approach makes prompt development easier to test, maintain, and improve systematically.

Key terms: source code, prompt engineering, prompt programming, clean, readable, testable, and modular

1.2.3 A methodical search for optimal prompts

The section explains why finding better prompts through ad hoc experimentation is slow and why apparent improvements need rigorous testing.

• Task-dependent prompting heuristics: Explain why a prompting heuristic cannot be assumed to improve a prompt for a particular task, and give an example of a technique the section discusses.
  Why it matters: A promising prompt technique is a candidate to test, not a guarantee of improvement.
• Large prompt search space: Explain why finding an optimal prompt through ad hoc prompt engineering can take so long and still miss good options.
  Why it matters: It explains why trying prompt variations by hand is an inefficient way to find strong candidates.
• Unreliable apparent prompt gains: Explain why a prompt may appear to improve during prompt engineering without having truly improved.
  Why it matters: It is a reason to verify prompt improvements rather than relying on a few encouraging results.

Key terms: heuristics, prompting techniques, search space, optimal prompts, ad hoc, small test sets, test rigorously

1.2.4 The bitter lesson

The section applies AI’s “bitter lesson”—that computational search and learning tend to outperform hand-coded expertise—to the development of prompts.

• Computation Over Encoded Expertise: Explain why general methods that use computation to search and learn tend to outperform systems built around human-coded domain knowledge, using chess as the section’s example.
  Why it matters: This principle explains why the section favors learning and search over relying solely on expert-designed solutions.
• The Lesson for Prompts: Explain how the section extends the bitter lesson to prompt development: an automated system allowed to search and learn is expected to outperform reliance on a person’s prompt-engineering expertise.
  Why it matters: It gives the reason for looking beyond prompts devised solely through personal expertise.

Key terms: the bitter lesson, general methods, computation, domain knowledge, prompt engineering, automated system, search and learn

1.2.5 A data-driven approach to tuning prompts

The section explains why reliable prompt comparison requires repeated testing and how DSPy makes that testing a reusable, data-driven process.

• Reliable prompt comparison: Explain why comparing prompts requires testing many inputs and running each prompt multiple times on each input.
  Why it matters: Repeated tests across varied inputs give a more dependable basis for choosing a prompt.
• Data-driven evaluation setup: Describe how collecting test data and defining a consistent evaluation function lets DSPy evaluate candidate prompts automatically, despite the difficulty of judging responses with different strengths.
  Why it matters: An explicit evaluation setup makes prompt experiments easier to compare and repeat.

Key terms: optimal prompt, inputs, stochastic, prompt programming, test data, evaluation function, candidate prompts

1.2.6 Adapting to change

The section explains why LM applications need to adapt after launch and how DSPy can reduce the cost of updating their prompts.

• Reasons applications need updating: Explain why a working LM application may need a different model or revised prompts, citing changes in model availability or constraints and changes in response-quality needs.
  Why it matters: A prompt that works today may need revisiting when the model or the application's needs change.
• Savings from prompt retuning: Explain why DSPy may save work beyond initial development when an LM application changes.
  Why it matters: The value of DSPy extends to maintaining an application as its models and requirements change.

Key terms: LM, prompts, edge cases, prompt engineering, DSPy, retune the prompts automatically

1.2.7 Making the best use of LMs

The section explains how automated prompt search can make smaller language models practical despite their sensitivity to prompts.

• Small-model prompting tradeoff: Explain why developers may choose larger LMs for complex problems even when smaller ones could cost less to run.
  Why it matters: This tradeoff explains why improving prompt search can change which model is practical to deploy.
• Automated search for smaller LMs: Explain how prompt programming with DSPy can make smaller LMs more practical for production.
  Why it matters: A better prompt search may let you deploy a smaller model rather than pay for a larger one.

Key terms: smaller LMs, prompts, long-term costs, prompt programming, DSPy, optimal prompt

1.3 Working with complex LM-based applications in DSPy

This section explains why complex LM applications are difficult to tune manually and how DSPy helps compose and optimize their workflows.

• Prompt tuning at scale: Explain why manually tuning prompts becomes impractical in large LM applications and why response cost and speed matter.
  Why it matters: The scale of the calls explains why improving prompts one at a time is not enough.
• Modular application control flows: Describe how a complex application's control flow can depend on earlier LM or tool results, and how DSPy's modular organization helps represent it.
  Why it matters: Thinking in connected steps makes a complex application easier to describe and build.
• Whole workflow prompt optimization: Explain what DSPy optimizes after a workflow is defined and what result it seeks.
  Why it matters: Optimizing prompts together is especially valuable when a workflow has many interconnected calls.
• Compounding errors in agents: Justify the need for reliable prompts and rigorous evaluation in complex applications, especially agents.
  Why it matters: Evaluating response quality matters more when mistakes can propagate or cause actions.

Key terms: prompt tuning, LM calls, query costs, control flows, modular organization, workflows, automatically optimize, set of prompts, overall results, reliable prompts

1.4 Where is DSPy useful?

This section explains when DSPy is worth using and when direct prompting may be sufficient.

• DSPy application fit: Explain why DSPy is especially useful for large or demanding LM applications, while still being useful for a single-request application.
  Why it matters: This helps you recognize that DSPy is not reserved for complicated applications.
• Direct prompting tradeoff: Describe when manually prompting an LM may be enough instead of using DSPy.
  Why it matters: Not every LM interaction warrants adopting a programming tool.

Key terms: DSPy, prompt programming, LM requests, directly prompting, manually created prompts

1.5 Working with LangChain

This section compares LangChain and DSPy, explaining their overlap, their different approaches to prompts and workflows, and when using both may help.

• LangChain and DSPy capabilities: Distinguish what LangChain and DSPy both support from their difference in prompt creation and optimization.
  Why it matters: The distinction helps you choose a framework without assuming that shared workflow features imply shared prompt-optimization features.
• Complementary framework roles: Explain when using LangChain or LangGraph alongside DSPy can be useful, despite their overlap.
  Why it matters: Using both makes sense when each has a distinct job, not simply because an LM application needs two frameworks.
• Plain Python workflow rationale: Describe why the authors favor DSPy’s approach to defining workflows over LangChain’s abstractions.
  Why it matters: The workflow approach affects how easily you can express and maintain an application pipeline.

Key terms: LangChain, DSPy, prompts, workflows, LangGraph, plain Python, pipelines

1.6 Building LLM applications through baselines and optimization

This section presents a DSPy development workflow: build a simple baseline, evaluate it, optimize prompts if needed, and deploy when the application is satisfactory.

• Baseline evaluation optimization workflow: Describe the recommended DSPy workflow: create a simple baseline, evaluate its performance, then let DSPy optimize prompts for LM calls if improvement is needed. Explain that the baseline may already be sufficient and that a satisfactory application can go into production.
  Why it matters: Evaluating a baseline gives you evidence for deciding whether prompt optimization is necessary.

Key terms: baseline, evaluation, optimizing, prompts, production

1.7 What this book covers

The section outlines the book’s coverage and explains that real-world datasets support its emphasis on evaluation and iterative improvement.

• Evaluation and iterative improvement: Explain how the book uses real-world datasets to demonstrate evaluating applications and improving them iteratively, and why that process matters.
  Why it matters: It frames reliable LLM applications as something to evaluate and improve, not merely build once.

Key terms: real-world datasets, evaluating, iterative improvements, reliable systems

1.8 Summary