A rubric is a set of criteria and scoring standards for judging an output. Like a grading rubric, it makes a judgment more specific than “good” or “bad.” For an agent, the criteria describe what quality means; the scale describes how well the output meets them. The rubric also needs validation against human judgment: consistent scores are not useful if they consistently reward the wrong things. section 7.3.4
Use a rubric when several answers could be acceptable, as with explanations, plans, reports, or images. If an output has a known correct target, such as a classification label or extracted field, metrics such as accuracy may be simpler and more appropriate. One agent can use both approaches for different parts of its output. section 7.3.4
To design a useful rubric:
- State the output’s purpose and the audience or task it must serve. section 7.3.4
- Choose specific, assessable criteria aligned with that purpose. For a recommendation, these might include fit with the requested genre, subject, and format. section 7.3.4
- Define a scoring scale and a passing threshold, then check whether its judgments agree with human judgment. section 7.3.4 section 7.3.5
- Include behavior when it matters: supported claims, logical reasoning, correct tool use, and recovery from errors can matter as much as tone or completeness. section 7.3.5
Image-style critic
Suppose an image generator must follow guidelines for color, lighting, characters, icons, and mood. A critic agent receives the guidelines and rubric, scores the generated image, and returns a pass/fail result with feedback. That feedback says what to refine while preserving what worked. Generation and critique can repeat until the image passes or an attempt limit is reached. section 7.3.5 section 7.6
Score the image on a scale of 1 to 5 using the following scale: 1. **Poor**: The image does not meet the style guidelines at all. 2. **Fair**: The image meets some of the style guidelines but has significant issues. 3. **Good**: The image meets most of the style guidelines but has minor issues. 4. **Very Good**: The image meets all the style guidelines with only a few minor issues. 5. **Excellent**: The image meets all the style guidelines perfectly. Pass the image if it scores 2 or higher, otherwise fail it.
The five-point image-quality scale shows how scoring descriptions and a pass threshold turn style guidelines into an evaluation rule. section 7.3.5
Listing 7.12 08_image_vision_critic_agents.py (critic agent)
class CritqueImage(BaseModel): #1
"""Result of critiquing image."""
image_pass: bool
feedback: str
critic = Agent(
name="Image Critic",
instructions=f"""You are an image critic.
Your task is to evaluate the quality of generated images
based on the provided specific criteria and style guidelines.
{style_guidelines} #2
{rubric} #3
""",
model="gpt-5-mini",
tools=[describe_image], #4
output_type=CritqueImage, #5
)
# critic loop
while failing := True: #6
input = dict(
description=image_description,
feedback=feedback, #7
)
result = await Runner.run(agent, str(input))
# code to save the image ommited
critique_result = await Runner.run(
critic,
f"Please critique the image at {image_path} with the prompt: {image_description}",
)
critique = critique_result.final_output
failing = critique.image_pass is False #6
feedback = critique.feedback #7
The critic-agent listing shows the rubric and style guidelines in the evaluator’s instructions, with typed pass/fail and feedback fields. section 7.3.5
