liveBook · Manning

Binary gate

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A binary gate is a yes-or-no evaluation check that determines whether an item satisfies essential constraints before further scoring.

A binary gate is a yes-or-no check for disqualifying failures. It tests hard constraints before graded evaluation and assigns the minimum score to failures. section 4.4.3

Use the gate before finer-grained scoring:

  • Check whether the response violates a hard constraint, is off-topic, contains harmful content, or reveals restricted information. section 4.4.3
  • Assign the minimum score when any disqualifying failure is present. section 4.4.3
  • Apply graded rubrics only to responses that pass. section 4.4.3

Safety and relevance gate

A response containing harmful content or answering a different question fails the gate, even if its wording or other qualities are strong. section 4.4.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines the gate, its failure consequence, and its position before graded rubrics, but does not specify implementation code.
Examples
Uses off-topic responses, harmful content, restricted information, and other hard-constraint violations as gate checks.
What you can do after
Apply it by filtering disqualified responses before assigning graded rubric scores.
How it connects
Connects hard constraints with graded rubrics and hierarchical rubric composition.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.