liveBook · Manning

Security boundary testing

As explained in Machine Learning Platform Engineering

Security boundary testing evaluates whether an AI application appropriately restricts responses and access at sensitive security boundaries.

Security boundary testing checks whether an AI application deflects security-sensitive requests without revealing protected information, while redirecting legitimate users to appropriate channels. section 13.2.1

A production-oriented test should:

  • Evaluate intent and response behavior rather than matching fixed refusal phrases. section 13.2.1

G-Eval boundary test

Define a G-Eval metric named “Security Boundary Respect,” pass the request and actual output as evaluation inputs, and require a 0.9 threshold. The evaluator judges whether the response preserves the boundary regardless of its exact wording.

How Machine Learning Platform Engineering covers it

How deep it goes
Defines semantic criteria, test inputs, outputs, and a threshold, but does not specify boundary enforcement implementation.
Examples
Uses a request for DataKrypt API keys and passwords, then evaluates refusal, protection, and redirection behavior.
What it assumes
Assumes an LLM application can produce test responses and requires familiarity with G-Eval test cases and evaluation criteria.
What you can do after
Diagnose whether responses appropriately deflect credential-seeking requests without relying on exact refusal wording.
How it connects
Connects G-Eval with enterprise RAG, functional correctness, response deflection, and LLM-as-judge evaluation.
Cover of Machine Learning Platform Engineering
liveBookmanning.com
Read Machine Learning Platform Engineering on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Platform Engineering. The section links point to the parts of the book it draws on.