liveBook · Manning

GitHub Actions

As explained in Enterprise RAG

GitHub Actions is a GitHub automation service for running software-development workflows in response to repository events or on schedules.

GitHub Actions turns a push into a repeatable delivery gate for a RAG application. A workflow in .github/workflows defines the runner, runtime, dependency installation, tests, secrets, and—when the checks pass—the path to a deployed container. This replaces manual rebuilding, docker pushes, and restarts with a pipeline that can catch broken changes before users receive them. section 5.3 section 5.5

A minimal evaluation workflow runs whenever code is pushed to main. It uses an Ubuntu runner, checks out the repository, sets up Python 3.11, installs pinned requirements, exposes OPENAI_API_KEY from repository secrets, and runs pytest. The tests send evaluation questions to the RAG application, compare generated answers against expected answers, and require a PASS result. section 2.4.3

Build a production-oriented workflow in stages:

  • Store dependencies in requirements.txt and pin the OpenAI package version so automated runs use a stable dependency set. section 2.4.3
  • Place workflow YAML under .github/workflows and trigger the test job on pushes to main. section 2.4.3
  • Use checkout, Python setup, dependency installation, and pytest steps to make the runner reproduce the test environment. section 2.4.3
  • Keep credentials in GitHub Secrets rather than source code; inject them as environment variables only for the step that needs them. section 2.4.3 section 5.2.4
  • Bind a deployment workflow to a GitHub environment so tests can access the live credentials needed for Cosmos DB, embeddings, and search-index checks. section 5.2.4
  • Run tests inside the container image intended for production, making a passing result evidence that the image includes required libraries as well as working logic. section 5.2.4

Automatic RAG evaluation on push

A test loops over questions such as “What is Underwhelming Spatula?”, obtains each RAG answer, and asks an LLM evaluator whether it matches the expected information. The GitHub Actions job runs pytest after installing openai, dotenv, and pytest; a missing PASS fails the pipeline. As requirements grow, the evaluator prompt can demand exact dates or numbers and explanations for failures. section 2.4.3

For deployment, GitHub Actions can rebuild the container and rerun tests after each commit. Only a passing run swaps the new image into the Azure Web App, creating a short feedback loop from code change to tested release. Logs and later load testing complement this gate when diagnosing behavior under real traffic. section 5.3 section 5.5

Figure 2.4 The automated eval system for RAG development. Evals compare chatbot answers to expected outputs and automate testing via GitHub Actions, ensuring high-quality performance.

The diagram links expected-answer evaluations, RAG responses, automated tests, and the GitHub Actions gate in one delivery flow. section 2.4.3

How Enterprise RAG covers it

How deep it goes
Details workflow YAML, runners, Python setup, dependency installation, secrets, pytest, and deployment gating without explaining GitHub internals.
Examples
Uses a main-branch pytest workflow for LLM-scored RAG answers and a container deployment pipeline for an Azure Web App.
What it assumes
Assumes a GitHub repository, Python tests, requirements.txt, and available service credentials; teaches the workflow configuration from files and settings.
What you can do after
Operate/configure it to test RAG changes on push and release only passing container builds.
How it connects
Connects CI/CD, pytest, repository secrets, Docker containers, Azure Web Apps, Cosmos DB, and Azure AI Search.

Also explained in

Compare how 11 books explain GitHub Actions →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Enterprise RAG. The section links point to the parts of the book it draws on.