MultiChainComparison examines predictions and their reasoning from two or more programs, then produces one final prediction. Unlike BestOfN, its answer can be new or combine earlier predictions. section 7.3.3
Its implementation follows a simple pattern:
- Collect predictions first; they commonly come from ChainOfThought programs, but any program with a reasoning field can supply them. section 7.3.3
- Use one Predict submodule and one language-model call to compare the collected predictions. The example selects the last output field; handling multiple output fields requires a custom module. section 7.3.3
Listing 7.8 Simplified version of MultiChainComparison
from dspy import InputField, OutputField, Predict
from dspy.primitives.module import Module
from dspy.signatures.signature import Signature, ensure_signature
class MultiChainComparison(Module):
def __init__(self, signature, M=3, temperature=0.7, **config):
super().__init__()
self.M = M
signature = ensure_signature(signature) #1
*_, self.last_key = signature.output_fields.keys() #2
for idx in range(M): #3
signature = signature.append(
f"reasoning_attempt_{idx+1}",
InputField(desc="${reasoning attempt}"),
)
signature = signature.prepend( #4
"Rationale", OutputField(desc="${corrected reasoning}"),)
self.predict = Predict(signature, temperature=temperature, **config) #5
def forward(self, completions, **kwargs):
attempts = [] #6
for c in completions: #7
rationale = c.get("rationale",
c.get("reasoning")).strip().split("\n")[0]
answer = str(c[self.last_key]).strip().split("\n")[0].strip()
attempts.append(
f"«I'm trying to {rationale} I'm not sure but my prediction
is {answer}»",
)
kwargs = {
**{f"reasoning_attempt_{idx+1}": attempt for idx, attempt in
enumerate(attempts)},
**kwargs,
}
return self.predict(**kwargs) #8
lm = dspy.LM("openai/gpt-4o-mini", api_key=OPENAI_API_KEY, cache=False)
dspy.configure(lm=lm)
qa_signature = "question -> answer"
base_prog = dspy.ChainOfThought(qa_signature)
question = "What is a good way to find an apartment?"
pred1 = base_prog(question=question)
pred2 = base_prog(question=question)
pred3 = base_prog(question=question)
prog = MultiChainComparison(signature=qa_signature, M=3)
prog(completions=[pred1, pred2, pred3], question=question)
A simplified Python implementation shows how to construct the signature and Predict submodule. section 7.3.3
