A deployment pipeline automates steps that move software or models toward production. For ML, it can retrain and deliver models as new data arrives; for application code, it can test changes, build an image, and trigger deployment. section 2.5.2 section 3.4.1
A GitLab CI pipeline for a FastAPI application uses three stages:
- Test the Python application with unit tests. section 3.4.1
- Build a Docker image and push it to Docker Hub. section 3.4.1
- Update the Helm chart with the new image tag. section 3.4.1
Listing 3.13 GitLab CI example
image: docker:20.10.16
variables:
DOCKER_TLS_CERTDIR: "/certs"
stages: #1
- test
- build
- update
services: #2
- docker:20.10.16-dind #3
test_code: #4
stage: test
image:
name: python:3.10
script:
- echo "testing code"
- pip install -r $CI_PROJECT_DIR/requirements.txt
- pytest $CI_PROJECT_DIR/
build_image: #5
stage: build
script:
- echo "building docker image"
- cd $CI_PROJECT_DIR
- echo ${CI_COMMIT_SHA:0:8}
- docker build . -t varunmallya/hello-joker:${CI_COMMIT_SHA:0:8}
- docker login -u ${DOCKER_USERNAME} -p ${DOCKER_PASSWORD}
- docker push varunmallya/hello-joker:${CI_COMMIT_SHA:0:8}
update_helm_chart: #6
stage: update
image:
name: python:3.10
script:
- apt-get update && apt-get install git
- git clone
https://${GITLAB_USER_NAME}:${GITLAB_ACCESS_TOKEN}@\
gitlab.com/learn-mlops/gitlab-ci-example.git #7
- cd $CI_PROJECT_DIR
- wget
https://github.com/mikefarah/yq/releases/download/v4.2.0/\
yq_linux_amd64 -O /usr/bin/yq && chmod +x /usr/bin/yq #8
- git config --global user.email "${GITLAB_USER_EMAIL}"
- git config --global user.name "${GITLAB_USER_NAME}"
- yq e -i ".image.tag |= "${CI_COMMIT_SHA:0:8}"" \
hello-joker/values.yaml #9
- git add hello-joker/values.yaml
- git commit -m "[skip ci]Update helm chart" #10
- git push https://${GITLAB_USER_NAME}:\
${GITLAB_ACCESS_TOKEN}@gitlab.com/learn-mlops/\
gitlab-ci-example.git HEAD:main #11
The configuration shows the stages, jobs, images, and scripts used to automate testing, image building, and chart updates. section 3.4.1
