A reward function converts a model response into a numerical score. In reinforcement fine-tuning, it grades sampled outputs so the policy can be updated to favor higher-scoring responses. When the answer is verifiable, a programmatic grader can replace human preference annotations and reward-model training. section 8.5.2 section 8.5
Match the grader to what can be checked:
- For structured extraction, parse the response and compare required fields with known values; return zero if parsing fails. section 8.5.2
- For math or code, compare the final answer with ground truth or run generated code against test cases. section 8.5.2
- For multi-step tasks, score the whole trajectory when the workflow and its evaluation criteria can be represented programmatically. section 8.5.1
Invoice extraction
A verifier checks an extracted invoice number, amount, and due date against expected values. It returns the fraction of matching fields, or zero for an unparseable response: a fully correct response scores 1.00, one wrong field scores 0.67, and prose that is not JSON scores 0.00. section 8.5.2
Listing 8.5 verifiable_reward.py (key sections)
# (key sections -- full source in code/chapter08/scripts/verifiable_reward.py)
REQUIRED_FIELDS = ("invoice_number", "amount", "due_date")
def invoice_reward(response: str, expected: dict) -> float: #A
if parsed is None: #B
return 0.0
hits = sum(1 for f in REQUIRED_FIELDS
if str(parsed.get(f, "")).strip()
== str(expected[f]).strip()) #C
return hits / len(REQUIRED_FIELDS)Inspect the Python-style invoice verifier, which illustrates parsing, field-by-field comparison, and a numeric score. section 8.5.2
A useful reward is deterministic, inexpensive to evaluate, and aligned with the quality dimension that matters. GRPO requires a callable function that scores individual responses; its compute cost rises with the number of generated responses, making cheap checks such as string matching more practical than judge-model scoring. DPO, by contrast, does not require a separate reward model or RL loop. section 8.5.1 section 8.3.3 section 8.2


