Refine

As explained in Building LLM Applications with DSPy

Refine is a DSPy module that iteratively evaluates responses and uses feedback to improve subsequent outputs.

Refine retries a DSPy program, scoring each response with a reward function and using feedback to improve later attempts. It stops when a score exceeds a threshold or it reaches the iteration limit, then returns the best-scoring response. section 7.3.4

A typical Refine run:

• Wrap a DSPy program with Refine and provide a reward function, maximum iteration count N, and score threshold. section 7.3.4
• Score each response; if it misses the threshold, use model feedback as a hint for the next attempt. section 7.3.4
• Expect two model calls per iteration: one for the program response and one to generate feedback when continuing. section 7.3.4

Enforcing a one-word answer

For a question about Belgium’s economic capital, a reward function can return 1.0 for a one-word answer and 0.0 otherwise. Refine is configured to try up to three times and stop at a score of 1.0. section 7.3.4

Figure 7.1 The workflow within the Refine module starts with the original prompt, which is used in a call to the LM. The response is collected, and Refine calls the reward method to evaluate it. At each iteration, the LM is also called to get feedback about the prompt and the response, including advice on how to improve the prompt. When asking for this advice, Refine also provides the LM with task and reward function information, a history of the previous prompts, and the reward function’s scores for their outputs. This advice is used as the hint field in the next call to the LM. Once the response is good enough (the reward function gives it a score above a specified threshold) or the process loops a specified number of times, the best response (as evaluated by the reward function) is returned.

Follow the response-scoring and feedback loop, including how reward scores and prompt history inform later attempts. section 7.3.4

How Building LLM Applications with DSPy covers it

How deep it goes

Explains the retry loop, reward scoring, feedback calls, and stopping rules, but does not examine the full implementation.

Examples

Demonstrates a one-word reward for answering a question about Belgium’s economic capital.

What you can do after

Apply it via DSPy by wrapping a program with a reward function, iteration limit, and score threshold.

How it connects

Connects DSPy programs with reward functions, ChainOfThought, Predict, and BestOfN.

liveBookmanning.com

Read Building LLM Applications with DSPy on liveBook Get the book at manning.com

Related concepts

• reward functionuses · A reward function maps an action, output, or sequence of behavior to a numerical score representing its quality or performance.
• ReActuses · ReAct is a DSPy module that implements an agent loop in which a language model selects tools and uses their results to guide subsequent steps.
• thresholduses · A threshold is a boundary value against which a quantity is compared to determine whether a condition is met or an action or decision is triggered.
• modulekind of · A DSPy module is a reusable, composable component that implements language-model behavior or a workflow.
• refinementimplements · Refinement is an iterative process of revising an intermediate result to progressively improve its quality, consistency, or performance.

This explainer was generated by Manning's knowledge graph from the text of Building LLM Applications with DSPy. The section links point to the parts of the book it draws on.