Test-driven development writes tests before implementation, then builds code that satisfies them. For automated RAG evaluations, expected questions and answers become executable checks before the chatbot is completed. section 2.4.3
A practical evaluation-driven workflow is:
- Define evaluation questions and expected answers. section 2.4.3
- Generate chatbot answers, compare each with its expected answer, and assert PASS. section 2.4.3
- Run the checks locally and automatically on pushes to the main branch through GitHub Actions. section 2.4.3
RAG answer evaluation
A test asks who wrote “Dubious Parenting Tips,” expects “Lisa Melton,” generates an answer through the RAG system, and evaluates whether both answers provide the same information. section 2.4.3
Figure 2.4 The automated eval system for RAG development. Evals compare chatbot answers to expected outputs and automate testing via GitHub Actions, ensuring high-quality performance.
Trace the automated evaluation pipeline from expected outputs through chatbot comparison to GitHub Actions testing. section 2.4.3






















