A DSPy Example is a dictionary-like record for one case in a dataset. During evaluation, each record supplies a program’s inputs, captures its language-model response, and provides the fields needed to score that response. section 4.1.1
Include the fields the evaluation needs:
- Provide every input field defined by the program’s signature, such as message and labels. section 4.1.1
- Include output fields used for scoring, such as the true intent label; omit outputs that are not scored. section 4.1.1
- Use with_inputs() to identify which fields are sent to the language model. section 4.1.1
Listing 4.1 Creating a DSPy Example object
import dspy
example_1 = dspy.Example({
"message": "I would like to find a flight from charlotte to las vegas that
"makes at most one stop in between",
"labels": unique_intents,
"intent_label": "Flight Booking Request"}).with_inputs("message", "labels")
This flight-intent record pairs a message and candidate labels with the true intent, then marks the model inputs. section 4.1.1
Listing 4.2 Creating a test set as a list of Example objects
import dspy
test_examples = [
dspy.Example({
"message": "I would like to find a flight from charlotte to las vegas that
"makes at most one stop in between",
"labels": unique_intents,
"intent_label": "Flight Booking Request"}).with_inputs("message",
"labels"),
dspy.Example({
"message": "What time does my flight to charlotte arrive?",
"labels": unique_intents,
"intent_label": "Time Inquiry"}).with_inputs("message", "labels")
]
A test set is a list of Example records; the same pattern can turn a simple list of cases into records. section 4.1.1
