Chapter 2: Basic prompting and DSPy

Introduction

The introduction explains why software-driven LM applications need reliable prompts, notes that prompts often use task-dependent components, and explains why those components remain useful to understand when using DSPy.

• Reliable software-driven prompts: Explain why prompts used by software need more care than prompts entered interactively, and how that motivates prompt engineering or prompt programming.
  Why it matters: It explains why prompting an LM from an application takes more preparation than chatting with one.
• Task-dependent prompt components: Describe why application prompts often contain standard sections, giving examples while noting that the best selection varies.
  Why it matters: It gives a reason to recognize prompt components without treating one combination as universal.
• DSPy prompting abstraction: Explain why understanding prompt techniques remains useful even though DSPy usually avoids direct work with prompt text.
  Why it matters: It clarifies why the chapter discusses prompts before turning to DSPy code.

Key terms: LM calls, prompt engineering, prompt programming, standard sections, persona, demonstrations, DSPy, prompting techniques, prompt content

2.1 Prompts

The section contrasts editing prompt sections directly with specifying prompting techniques in DSPy.

• Prompt editing versus DSPy techniques: Distinguish direct prompt engineering from working with DSPy: the former changes prompt sections, while the latter specifies prompting techniques that roughly correspond to those sections.
  Why it matters: This distinction clarifies what you specify when using DSPy instead of editing a prompt yourself.

Key terms: prompt engineering, prompting techniques, sections

2.1.1 Classification prompt example

The section uses a customer-intent classification prompt to show how a structured prompt guides an LM, why ambiguous messages remain difficult, and how prompt engineering improves prompts manually.

• Structured prompt design: Explain that the example prompt uses deliberately phrased sections containing information needed for the task, without extra information that might sidetrack the LM.
  Why it matters: A prompt that looks natural can embody deliberate choices about what the LM needs to see.
• Intent classification choices: Describe what the example asks the LM to return and why a clear customer message is easier to classify than an unclear one.
  Why it matters: The example shows both the intended output and a limit of a well-formed prompt.
• Manual prompt improvement: Describe how prompt engineering improves a prompt when it lacks something the LM needs.
  Why it matters: A plausible first prompt may still need revision based on its results.

Key terms: prompt template, prompt, LM, customer intent, categories, Unknown, Update Email, prompt engineering, manual process

2.1.2 The components of an effective prompt

This section explains the purposes of common prompt sections and why their content, order, and wording should be chosen for the task and tested with the LM.

• Task-specific prompt design: Explain why an effective prompt includes the sections a task needs, in an appropriate order and wording, without unnecessary content. Describe why these choices must be checked with the LM being used.
  Why it matters: It keeps prompt design focused on what actually helps your task rather than on a fixed template.
• Task and response specifications: Distinguish the required task description from optional context, input/output descriptions, and output-format instructions, and describe what each contributes.
  Why it matters: Separating the task from its supporting instructions makes a prompt easier to specify clearly.
• Demonstrations and reasoning requests: Describe how demonstrations show the LM appropriate input–response pairs, distinguish one-shot from few-shot prompting, and explain what a chain-of-thought request asks for.
  Why it matters: Examples and reasoning requests communicate expectations that a task description alone may leave unclear.
• Additional prompt guidance: Describe the roles of constraints and a closer, and state how the section characterizes emotional appeals.
  Why it matters: These optional additions offer different ways to steer or reinforce a prompt without being required sections.

Key terms: prompt sections, prompt engineering, prompt programming, task description, context, input/output description, output format description, demonstrations, one-shot prompting, few-shot prompting

2.2 A full DSPy application

This section contrasts a full DSPy application with a simpler question-answering example and explains what a callable DSPy program does.

• Application versus prompt alone: Explain why listing 2.1 does more than the earlier classification prompt, even though its code is not shorter: it generates and sends a prompt, parses the response, and includes prompt optimization.
  Why it matters: This explains what the DSPy code contributes beyond writing a prompt by hand.
• Minimal question answering example: Describe the simpler application in listing 2.2: it specifies an LM and API key, defines a question-to-answer task, supplies a question, and displays the answer without optimizing the prompt.
  Why it matters: The simple example separates what is needed to get started from the optional optimization shown in the fuller application.
• Callable DSPy programs: Explain that Predict is a DSPy module, an instantiated module is called a program here, and calling that program with a question makes it generate a prompt, call the LM, and return a response.
  Why it matters: This identifies the component doing the work when the example is run.

Key terms: DSPy application, prompt, optimization, LM, API key, question -> answer, Predict, module, program, prediction

2.3 The main concepts in DSPy

This section identifies DSPy’s core abstractions and introduces how optimization and evaluation relate to them.

• Core DSPy abstractions: Explain that DSPy’s concepts are abstractions that let users work at a higher level, and name the LM, signature, module, and prediction as the four strictly necessary components.
  Why it matters: These four concepts provide the vocabulary for understanding the DSPy application introduced earlier.
• Optimization and evaluation roles: Describe optimization at the level given here: DSPy explores variations of a starting prompt in search of an optimal one, with metrics and examples supporting that process. State that evaluations estimate the strength of an LM-based application.
  Why it matters: It distinguishes the components needed to use DSPy from the concepts that help improve and assess an application.

Key terms: abstraction, LM, signature, module, prediction, optimization, metrics, examples, evaluations, optimizers

2.4 The language model

This section explains how to select and configure the language model DSPy uses for subsequent calls.

• LM selection and configuration: Explain that a DSPy LM object represents a language model, is created with a model name and API key, and is passed to dspy.configure() to select the model for subsequent calls.
  Why it matters: It explains how a DSPy application chooses the model its calls will use.

Key terms: LM, LM object, dspy.configure(), API key

2.4.1 Storing API keys in .env files

This section explains why LM API keys should be kept out of source code and how DSPy applications can obtain them from a .env file or Google Colab secrets.

• API key security and purpose: Explain why LM providers require API keys and why hardcoding those keys in source code is unsafe.
  Why it matters: Keeping keys out of shared code helps prevent unauthorized usage and charges.
• Local dotenv key loading: Describe how to keep provider keys in a local .env file and make them available to the running Python program.
  Why it matters: This lets a script use keys without placing them in the code or committing them to a repository.
• DSPy provider key lookup: Explain when DSPy can find an API key without an explicit api_key argument.
  Why it matters: It explains why later LM objects can be created without repeating a secret in their code.
• Google Colab secret alternative: Describe the section’s alternative for supplying keys when working in Google Colab.
  Why it matters: Colab users have a way to supply credentials without hardcoding them.

Key terms: API key, authentication, hardcoded strings, python-dotenv, .env, load_dotenv(), os.environ, .gitignore, api_key, OPENAI_API_KEY

2.4.2 Calling LMs directly

This subsection shows how to call a DSPy LM directly to test handcrafted prompts and confirm that the LM and API key work.

• Direct LM configuration check: Explain how passing a prompt directly to the LM object provides a quick check that the LM is accessible and the API key is configured properly, without using the usual DSPy components.
  Why it matters: A simple direct call helps you check your setup before building on it.
• Direct call input formats: Describe the two direct-call options shown: passing a handcrafted prompt or passing a dictionary with a role, prompt, and optional prior messages in the model’s expected format.
  Why it matters: These input options let you test either a simple prompt or a prompt with message context.

Key terms: LM object, API key, prompt, handcrafted prompts, role, prior messages

2.4.3 Using LiteLLM to access LMs

This section explains how LiteLLM lets DSPy access many LM providers through a common interface and what varies when selecting a provider and model.

• LiteLLM provider interface: Explain how LiteLLM lets DSPy work with LMs from different providers without changing most DSPy API calls.
  Why it matters: You can consider different LM providers without rewriting the rest of your DSPy calls.
• Provider access requirements: Describe what must be supplied for a provider and where to check which providers are supported.
  Why it matters: A common interface does not remove provider-specific credentials or guarantee that every listed LM is available.
• DSPy model name formats: Distinguish when a model name alone is sufficient from when DSPy needs a provider-prefixed LM name.
  Why it matters: The right LM name format helps DSPy identify the model you intend to use.

Key terms: LiteLLM, standard API, OpenAI-like interface, API key, LM providers, LM provider, LM name, mistral/mistral-tiny, gpt-4o-mini

2.4.4 LM caching

This section explains how DSPy caches LM responses by default and when and how to disable caching.

• Default LM response caching: Explain what happens when a DSPy program repeatedly calls the same LM with the same input, and why the default behavior is useful.
  Why it matters: A fast, repeated answer may come from the cache rather than a new LM call.
• Disabling caching for tests: Describe why caching should be disabled for response-consistency or execution-time tests, and how the LM-level and global settings control it.
  Why it matters: Disabling the cache lets a test observe fresh LM calls instead of repeatedly measuring or inspecting a stored response.

Key terms: cache, providers’ responses, LM, cache parameter, global caching settings, consistency, execution times

2.4.5 Setting LM parameters

This section explains how temperature and max_tokens affect an LM’s responses, and notes that some models require specific parameter values.

• Temperature and response variation: Explain how temperature affects the consistency and creativity of LM responses, and choose a lower or higher value based on the task.
  Why it matters: Temperature helps you match response variation to the task without assuming it guarantees a particular result.
• Token limits and complete answers: Explain what max_tokens is useful for, why a low limit can cut off an answer, and why stating the desired output size in the prompt is usually preferable.
  Why it matters: A response limit controls how much can be returned, but a prompt can tell the LM what complete answer to aim for.
• Model-specific parameter requirements: State that some LMs require particular parameter values when the LM object is created, using the section’s GPT-5 example.
  Why it matters: A model’s requirements can constrain which parameter values you set.

Key terms: temperature, LM, cache, max_tokens, prompt, GPT-5, reasoning models

2.4.6 Switching between LMs

This section explains how DSPy switches among language models and why an application might use more than one.

• LM selection and switching: Explain that a DSPy application can create multiple LM objects but uses exactly one LM at a time, and name the three ways the section gives to switch the LM.
  Why it matters: You can change which model DSPy uses without treating all the LM objects as active at once.
• Reasons to switch models: Describe the purposes the section gives for using different LMs across or within tasks.
  Why it matters: The choice of LM can reflect the task’s needs, cost, or a desire for varied answers.

Key terms: LM object, configure(), set_lm(), context manager, different LMs, different tasks, diverse set of potential answers

2.5 Signatures

This section explains how DSPy signatures specify an LM task through named inputs and outputs, and why they are a concise alternative to writing full prompts.

• Signature inputs and outputs: Explain how a DSPy signature states what the LM receives and what it should return, and how DSPy turns that specification into a prompt.
  Why it matters: A signature lets you specify the task without writing the full prompt yourself.
• Descriptive signature field names: Describe how to choose names for signature inputs and outputs, using `poem -> descriptive_essay` as an example.
  Why it matters: Clear field names help convey the intended task in a short signature.
• Signatures versus full prompts: Distinguish a concise DSPy signature from a longer handwritten prompt and from a conventional function definition.
  Why it matters: This distinction explains why working with DSPy can feel more like specifying a program than tweaking prompt wording.
• Output field ordering: Explain why the order of output fields can matter when a signature requests both reasoning and an answer.
  Why it matters: Output fields do more than name results: their order can affect what the LM produces.

Key terms: signature, inputs, outputs, prompt, descriptive_essay, input, output, function definition, output fields, reasoning

2.5.1 Example asking for a confidence score

Shows how a DSPy signature can request a confidence score alongside an answer and specify that the score should be numeric.

• Confidence as an output field: Describe how to request a confidence score alongside an answer by adding confidence to the signature’s output fields.
  Why it matters: The signature states which outputs the application asks DSPy to produce.
• Numeric confidence field type: Explain why and how to specify a type for confidence in the signature, and interpret the example’s value of 1.0.
  Why it matters: Declaring the field’s type distinguishes a numeric confidence score from the default string output.

Key terms: confidence, output fields, signature, strings, numeric value, type

2.5.2 Summarization example

This subsection uses a poem-summarization example to show how a DSPy program’s input and output follow its signature, while noting a limit on interpreting the result.

• Summarization signature fields: Explain that the call to `prog` supplies the poem text through the signature’s `poem` input field and that the response contains the signature’s `summarization` output field.
  Why it matters: The example makes the connection between a signature and a program call concrete.
• Familiar text knowledge caveat: Explain why a summary of this familiar poem might not be based entirely on the supplied stanzas.
  Why it matters: A plausible summary does not by itself show that the model relied only on the provided text.

Key terms: prog, signature, input field, poem, output field, summarization, LM, general knowledge, text that was passed in

2.5.3 Translation example

This subsection illustrates translation as a task that requires changing only the DSPy signature and notes what the example returned.

• Translation through signature changes: Explain that the translation example changes only the signature to translate text from The Rime of the Ancient Mariner into French, while the quality of the returned translation may depend on the LM.
  Why it matters: The example shows how a signature change can express a different task without changing the rest of the application.

Key terms: signature, translation, gpt-4o-mini

2.5.4 Entailment example

The section uses a Socrates example to show that entailment depends on what premises establish, not whether a conclusion happens to be true, and notes how to request a binary answer.

• Truth versus logical entailment: Explain why the true statement “Socrates lived in Greece” does not follow from the premises that Socrates is a man and all men are mortal.
  Why it matters: It keeps you from treating a fact as a valid inference when the premises do not establish it.
• Binary entailment response: Describe how changing the signature can request a simple binary response to the entailment test.
  Why it matters: The signature lets you specify the response format you want for the test.

Key terms: entailment tests, premises, signature, logically_valid: bool

2.5.5 Style transfer example

The section uses a poem rewrite to show how a DSPy signature specifies a style-transfer task and how the same approach extends to other LM tasks.

• Style transfer signature: Describe how the style-transfer signature specifies the inputs and expected outputs in the poem example.
  Why it matters: It shows how the task is expressed without handcrafting a full prompt.
• Signatures across simple tasks: Explain how the section extends the signature approach to other LM tasks, while distinguishing that claim from a guarantee of output quality.
  Why it matters: It separates the ease of specifying a task from how well the LM performs it.

Key terms: signature, input fields, style transfer, signatures, LM

2.6 Modules

This section explains how DSPy modules turn signatures into prompts, execute LM calls, and let developers change prompting techniques without rewriting the task definition.

• Signatures and modules: Distinguish a signature’s input-and-output task definition from a module’s choice of how to prompt the LM. Describe how a module takes a signature to create a DSPy program.
  Why it matters: Keeping the task separate from the prompting technique makes DSPy programs easier to change.
• Module call execution: Trace what happens when a DSPy program is called, using Predict as the example, and explain the role of forward().
  Why it matters: A program call does more than send text to an LM: the module also handles the response.
• Independent component changes: Explain how separating signatures from modules allows a prompting technique to change without changing the task definition, while noting that a module still needs to suit the task.
  Why it matters: You can try a different way of prompting without rebuilding the task specification, provided the technique fits the task.
• Manageable component selection: Describe the components specified for a DSPy application and how the section proposes choosing among alternatives.
  Why it matters: Separating the components turns prompt development into a tractable comparison of alternatives.

Key terms: signature, module, DSPy program, Predict, forward(), DSPy-created prompt, ChainOfThought, ProgramOfThought, modular, LM

2.7 Predictions

This section explains what a DSPy Prediction contains and how to access its output fields.

• Prediction objects and fields: Explain that a Prediction is a Python object representing an LM’s output in the format specified by a signature, and describe how to access its output fields.
  Why it matters: It lets you retrieve individual LM outputs from a DSPy result.

Key terms: Prediction, signature, output fields, square brackets, dot notation

2.8 Code readability and code history

The section explains how DSPy makes LLM applications easier to read, change, and review than applications built around large prompts.

• DSPy code readability: Explain why DSPy code can be more concise and readable than code containing large, complex prompts, while distinguishing predictable code execution from randomness in the underlying LM.
  Why it matters: Readable code makes the application’s intent easier to see without untangling a long prompt.
• Clear changes and reviews: Describe why changes to DSPy signatures or modules are easier to interpret in code history, and state the questions a reviewer can focus on.
  Why it matters: A reviewer can assess the purpose of a change rather than try to infer the effect of an edited prompt clause.

Key terms: DSPy code, prompts, LMs, code history, signature, module, Predict, ChainOfThought, MultipleChainComparison

2.9 Summary