An input scanner examines a user prompt before language-model inference and blocks, flags, or modifies content that violates configured policies. It helps constrain a financial-services chatbot to approved customer interactions. section 10.4
A scanner pipeline can combine specialized checks:
- BanSubstrings detects configured unwanted strings, while BanCompetitors targets competitor names. section 10.4
- BanTopics restricts configured subjects, and Toxicity evaluates harmful or offensive language. section 10.4
- PromptInjection detects attempts to manipulate the model through hostile input. section 10.4
Listing 10.16 Defining input scanners
topics_list = ["politics", "violence", "aliens", "religion"] #1 competitors_names = [ #2 "Citigroup", "Citi", "Fidelity Investments", "Fidelity", "JP Morgan Chase and company", "JP Morgan", "JP Morgan Chase", "JPMorgan Chase", ] input_scan_substrings = BanSubstrings( #3 substrings=competitors_names, match_type=MatchType.STR, case_sensitive=False, redact=False, contains_all=False, ) inp_scan_ban_competitors = BanCompetitors( #4 competitors = competitors_names, redact = False, threshold = 0.1, ) inp_scan_ban_topics = BanTopics(topics=topics_list, #5 threshold=0.5) inp_scan_toxic = Toxicity(threshold=0.5) #6 inp_scan_injection = PromptInjection(threshold=0.2) #7 input_scanners = [ #8 input_scan_substrings, inp_scan_ban_competitors, inp_scan_ban_topics, inp_scan_injection ]
Configure scanner instances with topic and competitor lists, thresholds, matching behavior, redaction, and an ordered scanner collection. section 10.4
Listing 10.18 Defining safeguard function
def apply_safeguards(input_prompt, inp_scanners=input_scanners,
out_scanners=output_scanners):
llm_response_blocked = "I am sorry, but I can't help you with this;
this prompt is not allowed."
sanitized_prompt_input, results_valid_input, results_score_input = #1
scan_prompt(inp_scanners, input_prompt, fail_fast=False)
results = { #2
"input": {
"prompt": sanitized_prompt_input,
"validity": results_valid_input,
"scores": results_score_input,
},
"inference": {},
"output": {}
}
if any(not result for result in results_valid_input.values()): #3
print(f"\nPrompt \"{input_prompt}\" was blocked.\nscores:
{results_score_input}\n")
results["inference"]["response"] = llm_response_blocked
results["inference"]["status"] = "Blocked: Input"
return results
output = run_inference(sanitized_prompt_input) #4
results["inference"]["response"] = output
results["inference"]["status"] = "Success"
sanitized_response, results_valid_output, results_score_output = #5
scan_output(out_scanners, sanitized_prompt_input, output,
fail_fast=False)
results["output"]["response"] = sanitized_response
results["output"]["validity"] = results_valid_output
results["output"]["scores"] = results_score_output
if any(not result for result in results_valid_output.values()): #6
print(f"Blocked Output:\n\n{sanitized_response}\nscores:\n{results_score_output}\n")
results["output"]["status"] = "Blocked: Output"
else:
print(f"Output is safe:\n\n{sanitized_response}\n") #7
results["output"]["status"] = "Success"
return results
Apply the scanners before inference, collect validity and scores, and return a blocked response when any input check fails. section 10.4
Blocking a competitor reference
A prompt asking for financial-services companies besides Fidelity is passed to apply_safeguards and receives a blocked response.
