Summarization produces a shorter representation of a text. In language-model applications, judging a summary means balancing what it includes, whether its statements are correct, and how it is worded. section 8.1 section 8.0
Assess a summary by considering whether it:
- Preserves important content without omitting key information. section 8.1
- States information correctly and avoids unnecessary details. section 8.1
- Uses suitable wording and an appropriate level of detail. section 8.1
These criteria can be difficult to balance in a single evaluation score. The desired quality and the code used to measure it must be defined for the application. section 8.0
prog = dspy.Predict("poem -> summarization")
prediction = prog(poem=rime)
A DSPy Predict signature maps poem input to a summarization output. section 2.5.2
Summarizing a poem
A sample uses the first 20 stanzas of The Rime of the Ancient Mariner as input. Its output recounts the Mariner’s voyage and the consequences of shooting the Albatross. Because the language model may already know the poem, its answer may draw on general knowledge rather than only the supplied text. section 2.5.2
