DSPy treats prompting less like writing clever strings and more like engineering a small software system. You describe inputs and outputs, define predictors and modules, validate results, and compile the program against examples and a metric. section 7.3.3
A practical DSPy workflow is:
- Create a signature describing the task, input fields, and output fields. section 7.3.3
- Choose a predictor or generation strategy, such as chain of thought or retrieval. section 7.3.3
- Wrap the workflow in a module and define validation logic or an evaluation metric. section 7.3.3
- Compile against examples so the optimizer can improve the resulting prompt workflow. section 7.3.3
The approach is useful when an LLM application needs repeatable evaluation rather than one-off prompt tinkering. It can also target hallucination-prone tasks without fine-tuning, tokenization changes, or LoRA or DoRA adapters, although results still depend on data quality, metrics, and model behavior. section 12.2.5
Optimizing local-model math answers
A GSM8K workflow loads a quantized Llama3-8B-Instruct model, defines a question-and-answer signature, evaluates a zero-shot predictor, then compiles a chain-of-thought module with BootstrapFewShot. The reported evaluation rises from 14.5% to 74.5%, and the optimized program is saved. section 12.2.5
Listing 12.3 DSPy for math
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import BitsAndBytesConfig
import torch
import dspy
from dspy.datasets.gsm8k import GSM8K, gsm8k_metric
from dsp.modules.lm import LM
from dspy.evaluate import Evaluate
from dspy.teleprompt import BootstrapFewShot
model_name = "meta-llama/Meta-Llama-3-8B-Instruct"
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(
model_name,
device_map="auto",
quantization_config=quantization_config,
attn_implementation="sdpa",
)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True,)
gms8k = GSM8K()
gsm8k_trainset, gsm8k_devset = gms8k.train[:30], gms8k.dev[:100]
Inspect the concrete DSPy-for-math setup, including local model loading, quantization, dataset preparation, evaluation, and prompt optimization. section 12.2.5
