A prompt template standardizes the instructions and answer format sent to an LLM. In math evaluation, it asks for a solution and places the final result on a separate line in boxed form, making automatic checking more reliable. section 3.9
Implement the template as part of the evaluation workflow:
- Render fixed instructions together with the problem text and an explicit Answer field. section 3.9
- Generate the response through the existing text-generation wrapper, then extract the final candidate answer. section 3.9
- Normalize and grade the extracted answer with the verification pipeline, rather than treating formatting alone as correctness. section 3.10
- Compare template choices on a larger dataset: the short-output template can reduce token use but may lower accuracy, while no template may produce longer, correct reasoning. section 3.9
Math evaluation template
A rendered prompt identifies the model as a helpful math assistant, supplies a question, and requests the final result as \boxed{ANSWER}. A generated response may be concise and correctly formatted yet still mathematically incorrect, so extraction and grading remain necessary. section 3.9 section 3.10
Listing 3.13 Function to render a prompt template for math evaluation
def render_prompt(prompt):
template = (
"You are a helpful math assistant.\n"
"Answer the question and write the final result on a new line as:\n"
"\\boxed{ANSWER}\n\n"
f"Question:\n{prompt}\n\nAnswer:"
)
return template
Inspect the Python function that renders the fixed instructions, boxed-answer requirement, question, and answer prompt. section 3.9
