DSPy
DSPy is a Python framework for declaratively programming, composing, evaluating, and optimizing language-model applications and pipelines.
LLMs in Production
DSPy is a framework for programming and optimizing language-model prompts and pipelines through declarative modules and evaluation-driven optimization.
- How it is used
- Programs language-model workflows as evaluated, compilable systems for prompt optimization, retrieval, and tool augmentation.
- How deep it goes
- Defines signatures, predictors, modules, validation, compilation, and optimization, while deferring broader implementation details to documentation and notebooks.
- Examples
- Uses a local Llama3-8B-Instruct model, GSM8K math data, zero-shot prediction, and bootstrapped few-shot chain-of-thought optimization.
- What you can do after
- Builds, evaluates, compiles, and saves an optimized DSPy workflow for a local language model.
Building LLM Applications with DSPy
The programmatic interface for defining and executing DSPy language-model operations.
- How it is used
- Replaces repeated manual prompt rewriting with programmatic task definitions and evaluated prompt selection for language-model applications.
- How deep it goes
- Works through model configuration and a Predict call, then outlines baseline evaluation and iterative optimization without optimizing the introductory example.
- Examples
- Uses question answering with answer and confidence fields, a scored customer-service intent classifier, and a poem-rewriting signature.
- What you can do after
- Apply it via DSPy to define and run an LM operation, then evaluate a baseline before optimizing prompts when quality warrants it.
Read how Building LLM Applications with DSPy explains DSPy →
How Large Language Models Work
DSPy is a programming framework for composing language-model programs while separating model, vectorization, prompt, and task-logic definitions.
- How it is used
- Builds maintainable language-model applications by separating model, retrieval, prompting, and task logic.
- How deep it goes
- Demonstrates configuration and a simple RAG pipeline but stops short of a full DSPy tutorial.
- Examples
- Uses GPT-3.5, Wikipedia, ColBERTv2, three retrieved passages, and a generated answer in a minimal RAG program.
- What you can do after
- Build it from scratch by configuring replaceable model and retrieval components around a simple RAG task.
Related concepts
- cachinguses · Caching is the practice of storing reusable data or computation results so later access can avoid repeating retrieval or computation.
- LiteLLMuses · LiteLLM is an open-source software library and proxy that provides a unified interface for calling language models from multiple providers.
- signatureuses · A signature is a declarative specification of the expected input and output fields of a language-model task or operation.
- prompt engineeringuses · Prompt engineering is the deliberate design and refinement of instructions, context, examples, or other inputs to guide an AI model’s behavior or outputs.
- promptproduces · An input, typically containing instructions or context, supplied to an artificial-intelligence model to guide or elicit its output.
- optimizationuses · Optimization is the systematic selection or adjustment of values, actions, or system designs to improve a specified objective, often subject to constraints.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.