Unit tests make some software tasks verifiable: correctness has a definable answer, tests can check it automatically, and the criteria are objective. Code quality, readability, and efficiency may still require judgment. section 1.1.1
A test-based evaluation of generated code proceeds in four steps:
- Provide the function signature and docstring. section 1.1.1
- Generate the function body. section 1.1.1
- Run the code in a sandbox against the tests. section 1.1.1
- Record pass or fail according to whether all tests succeed. section 1.1.1
Listing 1.1 Verifiable rewards example (HumanEval)
def has_close_elements(numbers: List[float], threshold: float) -> bool:
“””Check if in given list of numbers, are any two numbers
closer to each other than given threshold.
>>> has_close_elements([1.0, 2.0, 3.0], 0.5)
False
>>> has_close_elements([1.0, 2.8, 3.0, 4.0, 5.0, 2.0], 0.3)
True
The doctests for has_close_elements show how input examples and expected Boolean results can check a function. section 1.1.1


































































