Testing your GitLab CI/CD pipeline

Jakob Oswald

Dieser Blogpost ist auch auf Deutsch verfügbar

Pipeline setup

When providing a GitLab CI/CD pipeline as a service or base to multiple development teams the pipeline code itself often resides in a separate repository. The development teams then include the pipeline in their .gitlab-ci.yml and use it. The example assumes the use of the pipeline in its entirety but the testing strategy could also be used to test single jobs in a more unit-testish way.

Our example consists of a pipeline repository and an application repository as shown below.

devops/ # group containing devops related projects
  pipeline # the pipeline project
apps/
  some-application # an application using the pipeline via include

pipeline/
  jobs/
    build.yml
    test.yml
    quality.yml
  pipeline.yml

include:
  - local: 'pipeline/jobs/build.yml'
  - local: 'pipeline/jobs/test.yml'
  - local: 'pipeline/jobs/quality.yml'

stages:
  - build
  - test
  - quality

include:
  - project: 'devops/pipeline'
    file: 'pipeline/pipeline.yml'
    ref: '1.0.0'

Testing the pipeline

If you now want to develop a feature in the pipeline, how do you make sure that the pipeline still works without rolling it out to your customers and having them discover you completely messed up the functionality?

The simple solution is to create a test project that uses your pipeline to validate it still works. This would look something like the following code.

devops/
  pipeline
  spring-boot-test-app # the new test app
apps/
  some-application

include:
  - project: 'devops/pipeline'
    file: 'pipeline/pipeline.yml'
    ref: 'main'

As you have noticed we changed the ref property from 1.0.0 which is referencing a git tag to main assuming this is the default branch in the pipeline repository.

Let’s automate running the test project’s pipeline whenever the main branch in the pipeline changes. We can easily do so by adding a pipeline to the pipeline repository that makes use of GitLabs downstream pipelines feature and the trigger keyword. This leaves us with the code below.

pipeline/
  jobs/
    build.yml
    test.yml
    quality.yml
  pipeline.yml
.gitlab-ci.yml # <- the pipeline within the pipeline repository

stages:
  - test-pipeline

test-spring-boot-app:
  stage: test-pipeline
  trigger:
    project: 'devops/spring-boot-test-app'
    strategy: depend
  # we only run on changes of our main branch
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

The flow now looks like this

Every time a commit on the main branch in the pipeline repository is made a CI/CD pipeline in the devops/spring-boot-test-app is triggered, which in turn makes use of the pipeline code in devops/pipeline on the main branch. The depend strategy in the trigger will even fail the upstream pipeline if the downstream pipeline fails.

Improving the idea

If we only test the main branch of the pipeline we still might encounter failures only after we have already added a new feature. So what we actually would like to do is to test a feature branch before we merge it to the main branch. This is surprisingly easy since we have the branch information at hand in predefined GitLab variables like CI_COMMIT_BRANCH and CI_COMMIT_REF_NAME and we can use a variable in the ref property of the include as well. There are some limitations to which variables can be used in includes but we can work with a project or group variable and - this is important - a trigger variable which we will call PIPELINE_REF_NAME.

We will have to declare the PIPELINE_REF_NAME as a project variable for the test app in devops/spring-boot-test-app setting it to main so by default the main branch of devops/pipeline will be included. Then we will pass the the pipelines current branch name to the downstream in the trigger.

stages:
  - test-pipeline

test-spring-boot-app:
  stage: test-pipeline
  variables:
    PIPELINE_REF_NAME: $CI_COMMIT_REF_NAME # <- we pass the current branch name to the downstream pipeline
  trigger:
    project: 'devops/spring-boot-test-app'
    strategy: depend

include:
  - project: 'devops/pipeline'
    file: 'pipeline/pipeline.yml'
    ref: $PIPELINE_REF_NAME # <- will use the variable passed by the trigger or the default of the project variable

Now the include in the downstream pipeline of the test project is dynamically resolved depending on the upstream branch name allowing us to test our changed pipeline code prior to merging it.

Conclusion

This obviously does not guard against everything and is still time and resource consuming but it will allow you to create some test projects for common cases and help to catch some bugs early on. The automation makes it easy to incorporate testing in your pipeline development flow just like you would do with other pieces of software as well and likely improves the quality of your product.