DSPy
DSPy is a Python framework for declaratively programming, composing, evaluating, and optimizing language-model applications and pipelines.
LLMs in Production
DSPy is a framework for programming and optimizing language-model prompts and pipelines through declarative modules and evaluation-driven optimization.
- How it is used
- Programs language-model workflows as evaluated, compilable systems for prompt optimization, retrieval, and tool augmentation.
- How deep it goes
- Defines signatures, predictors, modules, validation, compilation, and optimization, while deferring broader implementation details to documentation and notebooks.
- Examples
- Uses a local Llama3-8B-Instruct model, GSM8K math data, zero-shot prediction, and bootstrapped few-shot chain-of-thought optimization.
- What you can do after
- Builds, evaluates, compiles, and saves an optimized DSPy workflow for a local language model.
Building LLM Applications with DSPy
The programmatic interface for defining and executing DSPy language-model operations.
- How it is used
- Supports declarative language-model applications as an alternative to manual prompt engineering.
- How deep it goes
- Explains model configuration, task signatures, prediction and optimization workflow, with a simple unoptimized code example.
- Examples
- Uses question answering, customer-service intent classification and poetry style transfer to illustrate DSPy tasks.
- What you can do after
- Apply it via DSPy to configure a model, declare a prediction task and evaluate an application before optimization.
Read how Building LLM Applications with DSPy explains DSPy →
How Large Language Models Work
DSPy is a programming framework for composing language-model programs while separating model, vectorization, prompt, and task-logic definitions.
- How it is used
- Builds maintainable language-model applications by separating model, retrieval, prompting, and task logic.
- How deep it goes
- Demonstrates configuration and a simple RAG pipeline but stops short of a full DSPy tutorial.
- Examples
- Uses GPT-3.5, Wikipedia, ColBERTv2, three retrieved passages, and a generated answer in a minimal RAG program.
- What you can do after
- Build it from scratch by configuring replaceable model and retrieval components around a simple RAG task.
Context Engineering
DSPy is an open-source framework for programming language-model systems with declarative modules and signatures that can be optimized against metrics and examples.
- How it is used
- Replaces hand-written prompts with declarative task programs optimized using examples, feedback, and evaluation signals.
- How deep it goes
- Defines signatures and modules, demonstrates CoT and ReAct, and describes optimization, but does not detail optimizer internals.
- Examples
- Uses a math tutor with a calculator and an action planner optimized against an owner-matching metric.
- What you can do after
- Build it from scratch by defining a signature, selecting a module, and compiling a predictor against examples and a metric.
Also covered in
- AI-Powered Search
- Retrieval Augmented Generation, The Foundational IdeasIn early access (MEAP)
What to know first
- promptA prompt is text or other input supplied to an AI model to provide instructions, context, or other information that guides its response or generated output.
- similarity searchSimilarity search is a retrieval technique that finds or ranks items by their similarity to a query, often by comparing feature or vector representations under a similarity or distance measure.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- prompt engineeringPrompt engineering is the deliberate design and refinement of inputs, instructions, and context to guide an AI model’s behavior or outputs.
Related concepts
- cachinguses · Caching is the temporary storage of data or computed results for reuse, reducing the work or delay of retrieving or computing them again.
- LiteLLMuses · LiteLLM is an open-source software library and proxy that provides a unified interface for calling language models from multiple providers.
- signatureuses · A signature is a declarative specification of the expected input and output fields of a language-model task or operation.
- moduleuses · A DSPy module is a reusable, composable component that implements language-model behavior or a workflow.
- optimizationuses · Optimization is the systematic selection or modification of values, configurations, or implementations to improve a defined objective, often subject to constraints.
- prompt engineeringuses · Prompt engineering is the deliberate design and refinement of inputs, instructions, and context to guide an AI model’s behavior or outputs.
Questions about DSPy
What is DSPy?
DSPy is a Python framework for declaratively programming, composing, evaluating, and optimizing language-model applications and pipelines.
Which Manning books explain DSPy?
LLMs in Production, Building LLM Applications with DSPy, How Large Language Models Work and Context Engineering.
What should I know before learning DSPy?
Manning books that teach DSPy usually introduce prompt, similarity search, language model and prompt engineering first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



