The OpenAI API provides a programmatic path from Python applications to OpenAI models. A production-oriented workflow uses it to generate answers, compare them with expected outputs, and automate regression testing. section 2.4.3
A practical integration typically includes:
- Install the OpenAI library and keep the API key in an environment variable loaded from .env rather than source code. section 2.4.3
- Send a user message through chat completions, selecting a model such as gpt-4o and reading the returned message content. section 2.4.3
- Use a second API call to judge whether a generated answer matches an expected answer, then assert the result in pytest. section 2.4.3
Automated RAG evaluation
A test collects answers for questions about fictional books, sends each expected/generated pair to an evaluation function, and runs the checks in GitHub Actions whenever code reaches main. section 2.4.3
Figure 2.4 The automated eval system for RAG development. Evals compare chatbot answers to expected outputs and automate testing via GitHub Actions, ensuring high-quality performance.
Shows the evaluation loop connecting chatbot answers, expected outputs, automated tests, and GitHub Actions. section 2.4.3





