HarmBench is an open-source framework for measuring how likely language models are to generate harmful or unsafe content during automated red-teaming. section 13.4.2
A HarmBench evaluation follows three practical steps:
- Create test cases containing prompts for the attacks being examined. section 13.4.2
- Generate relevant model responses for those prompts. section 13.4.2
- Assess completions to quantify how many attacks produced harmful results. section 13.4.2
Harmful-generation evaluation
An illustrated evaluation uses AutoPrompt and AutoDAN against Llama 2 70B for a bleach-and-ammonia-mixing scenario, showing positive and negative examples.
# Run all compatible attacks against Llama 2 7B Chat using a SLURM cluster python ./scripts/run_pipeline.py --methods all --models llama2_7b --step all --mode slurm # Generate and evaluate completions using a SLURM cluster python ./scripts/run_pipeline.py --methods all --models baichuan2_7b,mistral_7b,llama2_70b --step 2_and_3 --mode slurm
Pipeline commands demonstrate running compatible attacks and generating or evaluating completions across models through a SLURM cluster. section 13.4.2
