Skip to the content

FAQ

Using GitLab CI to test CL projects

On this page

This page contains instructions on how to leverage GitLab CI to test your CL project against multiple Common Lisp implementations.

GitLab CI is a very flexible piece of software. As such, there is no One True Way of doing things. However, we recommend that you start your GitLab CI journey using GitLab CI templates from the CLCI project.

The CLCI project both gathers links to a variety of CI solutions for CL and develops templates for GitLab CI. The GitLab CI template documentation can be found at https://clci.common-lisp.dev/clci-gitlab-ci/. Its project repo is at clci/gitlab-ci.

Step 1: Write tests

In order to test your code, you must first write some tests! There are a wide variety of Common Lisp testing frameworks. For a recent review of them, see https://sabracrolleton.github.io/testing-framework.

Once you have chosen a testing framework and written tests, you should tie it into ASDF such that (asdf:test-system "foo") will run your tests. You framework should have instructions on how to do this. If it does not, see the ASDF best practices for guidance.

Additionally, it is best practice for your system's test operation to signal an error if tests fail.

Step 2: Write a script to run your tests

There are a large number of ways to write system definitions and run tests. So CLCI's GitLab templates do not attempt to guess how you want your tests to be run. Instead, you must provide a Common Lisp file that, when loaded with cl:load will run your tests and either signal an error on failure or exit the process with a non-zero exit code.

The templates default to looking for this script at scripts/ci-test.lisp in your repo. However, it can be overridden by setting the $CLCI_TEST_SCRIPT variable.

In the default configuration, this script will be loaded into a CL process that already has ASDF loaded (defaulting to the latest version), and your chosen dependency manager loaded (if any). Thus, for most projects the contents of this file will simply be:

(ql:quickload "foo/test")
(asdf:test-system "foo")

Step 3: Add continuous integration through .gitlab-ci.yml

Next instructions have to be added through GitLab's CI (Continuous Integration) platform to actually run your tests. The example below is a simple way to run your tests on up to eight CL implementations.

include:
  project: 'clci/gitlab-ci'
  ref: v2-stable
  file:
    - guarded-linux-test-pipeline.gitlab-ci.yml

variables:
  # Uncomment if you want to use Quicklisp as your dependency manager during
  # tests.
  #
  # CLCI_DEPENDENCY_MANAGER: "quicklisp"
  #
  # Uncomment if you have Git submodules that you want the runner to
  # automatically init and update for you. Submodules are sometimes used by
  # projects to bundle their test dependencies.
  #
  # GIT_SUBMODULE_STRATEGY: recursive
  #
  # Uncomment these lines if you want to test against Allegro, you have read
  # the Allegro express license
  # <https://franz.com/ftp/pub/legal/ACL-Express-20170301.pdf>, *and* your use
  # of Allegro Express does not violate the license. Alternatively, uncomment
  # and provide your own Docker image (or runner) that has Allegro installed
  # with your actual license.
  #
  # CLCI_TEST_ALLEGRO: "yes"
  # I_AGREE_TO_ALLEGRO_EXPRESS_LICENSE: "yes"
  #
  # Uncomment if you want to test on Clasp. It is disabled by default due to
  # significant upstream churn. It will be enabled by default after their next
  # release.
  #
  # CLCI_TEST_CLASP: "yes"


# This section is not strictly required, but prevents Gitlab CI from launching
# multiple redundent pipelines when a Merge Request is opened.
workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never
    - if: '$CI_COMMIT_BRANCH'
    - if: '$CI_COMMIT_TAG'

Be sure to read the comments and uncomment lines that apply to you.

The above uses the pipeline defined by the guarded-linux-test-pipeline.gitlab-ci.yml file to test against up to eight (at the time of writing) CL implementations:

  • ABCL
  • Allegro
  • CCL
  • Clasp
  • CLISP
  • CMUCL
  • ECL
  • SBCL

By default, the template uses Docker images from the cl-docker-images project. These images contain a precompiled CL implementation, as well as a number of commonly used foreign libraries.

The workflow section has nothing to do with CL at all. GitLab CI treats pipelines associated with Merge Requests differently than pipelines associated with a branch. A poor GitLab CI configuration would spawn two pipelines for Merge Requests: one associated with the MR and one associated with the branch. This workflow tells GitLab CI to not create a "branch pipeline" once a MR has been opened for the branch.

Step 4: Deploy and test your new pipelines

If the syntax of your .gitlab-ci.yml file is correct, a new build will run on every push.

Note that if you want to check that your .gitlab-ci.yml file has valid syntax, you can use GitLab's "CI Lint" application. This application is available in every project on the "Pipelines" page. For the "clo/cl-site" project, it's located at https://gitlab.common-lisp.net/clo/cl-site/-/ci/lint

Please note that the CLCI GitLab templates have more functionality than just testing. You can easily add pipelines to cut releases of your project when you push a tag or periodically update your pinned dependencies if you use CLPM. Additionally, you can easily add jobs that run arbitrary CL scripts besides the one you wrote in scripts/ci-test.lisp

For more information, please make sure to read https://docs.gitlab.com/user/project/pages/ and https://clci.common-lisp.dev/clci-gitlab-ci/.

Something wrong or missing here? Edit this page on GitLab.