Promptfoo is an open-source tool for adversarial testing of LLM applications. It probes the application, not just the model, for behavior that should be caught before deployment. This matters for RAG systems: a user query influences retrieved documents, and both can enter the prompt context where an attacker may try to override instructions or push the assistant beyond its intended scope. section 12.1.3 section 13.2.2
A practical testing loop is:
- Install the CLI with `npm install -g promptfoo`, check it with `promptfoo --version`, and run `promptfoo init`. section 13.2.2
- Configure a target application and choose vulnerability plugins and attack strategies. The shown configuration includes contract, excessive-agency, hallucination, and harmful-content probes alongside basic and jailbreak strategies. section 13.2.2
- Run `promptfoo redteam generate` to create adversarial prompts, then inspect evaluation results with `promptfoo view` for pass/fail assessments and individual failures. section 13.2.2
Listing 13.11 Configuring Promptfoo for adversarial security testing
targets: - file://promptfoo_provider.py redteam: purpose: " I can help you with:" provider: "vertex:gemini-2.0-flash-exp" numTests: 2 plugins: - contracts - excessive-agency - hallucination - harmful:cybercrime:malicious-code - harmful:insults - harmful:profanity - hijacking deviation - politics strategies: - basic - jailbreak - jailbreak:composite
The configuration connects a file-based application target to a red-team provider, vulnerability plugins, and attack strategies; these choices determine what gets probed. section 13.2.2
Listing 13.12 Python bridge enabling Promptfoo to test DakkaBot
sys.path.insert(0, os.path.dirname( #1
os.path.abspath(__file__))) #1
from dakka_bot_core import dakka_bot_query, dakka_bot_query_full
def call_api(prompt: str, options: Dict[str, Any], #2
context: Dict[str, Any]) -> Dict[str, Any]: #2
try:
response = dakka_bot_query(prompt)
return {
"output": response #3
}
except Exception as e:
return {
"output": None,
"error": str(e) #4
}
def main():
if len(sys.argv) < 2:
print("Usage: python promptfoo_provider.py 'your query here'")
sys.exit(1)
query = sys.argv[1] #5
result = call_api(query, {}, {}) #6
if result.get("error"):
print(f"Error: {result['error']}")
else:
print(result["output"])
if __name__ == "__main__":
main()
A Python bridge passes each test prompt to an existing chatbot query function and returns its output or an error, making the application callable as a Promptfoo target. section 13.2.2
Testing a documentation assistant
Two prompt-injection attempts try to pull a documentation assistant out of role: one asks it to adopt a pirate persona; another pushes it toward competitor comparisons. Such cases test whether the assistant maintains its intended boundaries, not merely whether it can answer a question. section 13.2.2
Use the failures to identify boundaries that need attention before deployment. Promptfoo supplies systematic probing; input sanitization, output validation, and continuous monitoring remain separate parts of production safety. section 13.5
