firecow/gitlab-ci-local

Tired of pushing to test your .gitlab-ci.yml?

Run gitlab pipelines locally as shell executor or docker executor.

Get rid of all those dev specific shell scripts and make files.

Table of contents

• Examples
  • docker-compose-nodejs
  • docker-swarm-php
  • docker-in-docker-build
  • docker-in-docker with a local registry
• Installation
• Convenience
  • CLI options
    • Via a file
    • Via environment variables
  • DotEnv file
  • Bash alias
  • Tab completion
  • Listing jobs
• Quirks
  • Tracked Files
  • Local Only
  • Home File Variables
  • Remote File Variables
  • Project File Variables
  • Decorators
  • Includes
  • Artifacts
  • Self Hosted Custom Ports
• Development
  • Scripts
• Creating single executable binaries from source

Installation

Linux based on Debian

Users of Debian-based distributions should prefer the the Deb822 format, installed with:

sudo wget -O /etc/apt/sources.list.d/gitlab-ci-local.sources https://gitlab-ci-local-ppa.firecow.dk/gitlab-ci-local.sources
sudo apt-get update
sudo apt-get install gitlab-ci-local

If your distribution does not support this, you can run these commands:

curl -s "https://gitlab-ci-local-ppa.firecow.dk/pubkey.gpg" | sudo apt-key add -
echo "deb https://gitlab-ci-local-ppa.firecow.dk ./" | sudo tee /etc/apt/sources.list.d/gitlab-ci-local.list

# OR

# MUST be `.asc` at least for older apts (e.g. Ubuntu Focal), since the key is ASCII-armored
PPA_KEY_PATH=/etc/apt/sources.list.d/gitlab-ci-local-ppa.asc
curl -s "https://gitlab-ci-local-ppa.firecow.dk/pubkey.gpg" | sudo tee "${PPA_KEY_PATH}"
echo "deb [ signed-by=${PPA_KEY_PATH} ] https://gitlab-ci-local-ppa.firecow.dk ./" | sudo tee /etc/apt/sources.list.d/gitlab-ci-local.list

# and then

sudo apt-get update
sudo apt-get install gitlab-ci-local

Note that the path /etc/apt/sources.list.d/gitlab-ci-local.list is used in the file gitlab-ci-local.list. If you change it in these commands you must also change it in /etc/apt/sources.list.d/gitlab-ci-local.list.

Arch Linux

gitlab-ci-local is available for Arch Linux through the AUR. Install with your favourite AUR helper.

paru -S gitlab-ci-local

NPM

npm install -g gitlab-ci-local

Bun

bun install -g gitlab-ci-local

Macos

bash version must be above or equal 4.x.x

brew install gitlab-ci-local

Windows (Git bash)

• Install gitbash
• Install rsync

Download and put binary in C:\Program Files\Git\mingw64\bin

curl -L https://github.com/firecow/gitlab-ci-local/releases/latest/download/gitlab-ci-local-windows-amd64.zip -o gcl.zip && unzip -o gcl.zip -d /c/Program\ Files/Git/mingw64/bin && rm gcl.zip

Executing gitlab-ci-local with --variable MSYS_NO_PATHCONV=1 can be useful in certain situations

Convenience

CLI options

[!NOTE] Most likely home-file-variables or project-file-variables is what you're looking for instead

All cli options can be assigned default values by either of the following ways:

Via environment variables

export GCL_NEEDS=true                   # --needs options
export GCL_FILE='.gitlab-ci-local.yml'  # --file=.gitlab-ci-local.yml

Via a file

# Either of the following:
# - `.gitlab-ci-local-env` in the current working directory
# - `$HOME/.gitlab-ci-local/.env`
NEEDS=true               # --needs
FILE=doctor-strange.yml  # --file=doctor-strange.yml

Bash alias

echo "alias gcl='gitlab-ci-local'" >> ~/.bashrc

Tab completion

gitlab-ci-local --completion >> ~/.bashrc

Logging options

export GCL_TIMESTAMPS=true # or --timestamps: show timestamps in logs
export GCL_MAX_JOB_NAME_PADDING=30 # or --maxJobNamePadding: limit padding around job name
export GCL_QUIET=true # or --quiet: Suppress all job output

List Pipeline Jobs

Sometimes there is the need of knowing which jobs will be added before actually executing the pipeline. GitLab CI Local is providing the ability of showing added jobs with the following cli flags.

--list

The command gitlab-ci-local --list will return pretty output and will also filter all jobs which are set to when: never.

name            description  stage   when        allow_failure  needs
test-job        Run Tests    test    on_success  false
build-job                    build   on_success  true           [test-job]
exit-codes-job               build   on_success  [42,137]       []
deploy-job                   deploy  on_success  [1]

• description: always shown, empty when not set
• allow_failure: true, false, or [exit_code1,exit_code2] when specific exit codes are allowed to fail
• needs: omitted when not specified (job follows stage ordering), [] when explicitly set to no dependencies (job starts immediately)

--list-all

Same as --list but will also print out jobs which are set to when: never (directly and implicit e.g. via rules).

name            description  stage   when        allow_failure  needs
test-job        Run Tests    test    on_success  false
build-job                    build   on_success  true           [test-job]
exit-codes-job               build   on_success  [42,137]       []
deploy-job                   deploy  on_success  [1]
never-job                    test    never       false

--list-csv

The command gitlab-ci-local --list-csv will output the pipeline jobs as a CSV-formatted list and will also filter all jobs which are set to when: never.

name;stage;when;allowFailure;needs
test-job;test;on_success;false;
build-job;build;on_success;true;[test-job]
exit-codes-job;build;on_success;[42,137];[]
deploy-job;deploy;on_success;[1];

• allowFailure: true, false, or [exit_code1,exit_code2] when specific exit codes are allowed to fail
• needs: empty when not specified (job follows stage ordering), [] when explicitly set to no dependencies (job starts immediately)

--list-csv-all

Same as --list-csv but will also print out jobs which are set to when: never (directly and implicit e.g. via rules).

name;stage;when;allowFailure;needs
test-job;test;on_success;false;
build-job;build;on_success;true;[test-job]
exit-codes-job;build;on_success;[42,137];[]
deploy-job;deploy;on_success;[1];
never-job;test;never;false;

Quirks

Tracked Files

Untracked and ignored files will not be synced inside isolated jobs, only tracked files are synced.

Remember git add

Local Only

local-only-job:
  rules:
    - { if: $GITLAB_CI == 'false' }

local-only-subsection:
  script:
    - if [ $GITLAB_CI == 'false' ]; then eslint . --fix; fi
    - eslint .

Home file variables

Put a file like this in $HOME/.gitlab-ci-local/variables.yml

---
project:
  gitlab.com/test-group/test-project.git:
    # Will be type Variable and only available if remote is exact match
    AUTHORIZATION_PASSWORD: djwqiod910321
  gitlab.com:project/test-group/test-project.git: # another syntax
    AUTHORIZATION_PASSWORD: djwqiod910321

group:
  gitlab.com/test-group/:
    # Will be type Variable and only available for remotes that include group named 'test-group'
    DOCKER_LOGIN_PASSWORD: dij3213n123n12in3

global:
  # Will be type File, because value is a file path
  KNOWN_HOSTS: '~/.ssh/known_hosts'
  DEPLOY_ENV_SPECIFIC:
    type: variable # Optional and defaults to variable
    values:
      '*production*': 'Im production only value'
      'staging': 'Im staging only value'
  FILE_CONTENT_IN_VALUES:
    type: file
    values:
      '*': |
        Im staging only value
        I'm great for certs n' stuff

Variables will now appear in your jobs, if project or group matches git remote, globals are always present

Remote file variables

You can also fetch variable files from remote git repositories. If you need multiple variable files, repeat the --remote-variables flag.

gitlab-ci-local --remote-variables git@gitlab.com:firecow/example.git=gitlab-variables.yml=master

Project file variables

The --variables-file [default: $CWD/.gitlab-ci-local-variables.yml] can be used to setup the CI/CD variables for the executors

yaml format

---
AUTHORIZATION_PASSWORD: djwqiod910321
DOCKER_LOGIN_PASSWORD: dij3213n123n12in3
# Will be type File, because value is a file path
KNOWN_HOSTS: '~/.ssh/known_hosts'

# This is only supported in the yaml format
# https://docs.gitlab.com/ee/ci/environments/index.html#limit-the-environment-scope-of-a-cicd-variable
EXAMPLE:
  values:
    "*": "I am only available in all jobs"
    staging: "I am only available in jobs with `environment: staging`"
    production: "I am only available in jobs with `environment: production`"

.env format

AUTHORIZATION_PASSWORD=djwqiod910321
DOCKER_LOGIN_PASSWORD=dij3213n123n12in3
# NOTE: value will be '~/.ssh/known_hosts' which is different behavior from the yaml format
KNOWN_HOSTS='~/.ssh/known_hosts'

Decorators

The @Description decorator

Adds descriptive text to gitlab-ci-local --list

# @Description Install npm packages
npm-install:
  image: node
  artifacts:
    paths:
      - node_modules/
  script:
    - npm install --no-audit

The @Interactive decorator

# @Interactive
interactive-shell:
  rules:
    - if: $GITLAB_CI == 'false'
      when: manual
  script:
    - docker run -it debian bash

The @InjectSSHAgent decorator

# @InjectSSHAgent
need-ssh:
  image: kroniak/ssh-client
  script:
    - ssh-add -L

The @NoArtifactsToSource decorator

Prevent artifacts from being copied to source folder

# @NoArtifactsToSource
produce:
  stage: build
  script: mkdir -p path/ && touch path/file1
  artifacts: { paths: [ path/ ] }

A global configuration is possible when setting the following flag

gitlab-ci-local --no-artifacts-to-source

Includes

Includes from external sources are only fetched once and cached. Use --fetch-includes to ensure that the latest external sources are always fetched.

Artifacts

Shell executor jobs copies artifacts to host/cwd directory. Use --shell-isolation option to mimic correct artifact handling for shell jobs.

Docker executor copies artifacts to and from .gitlab-ci-local/artifacts

Self Hosted Custom Ports

If your self-hosted GitLab instance uses custom ports, it is recommended to manually define the CI_SERVER_PORT and/or CI_SERVER_SHELL_SSH_PORT variables accordingly.

---
# $CWD/.gitlab-ci-local-variables.yml

CI_SERVER_PORT: 8443
CI_SERVER_SHELL_SSH_PORT: 8022

Special variables

• GCL_PROJECT_DIR_ON_HOST Absolute path to gitlab-ci-local current working directory on the host machine. Use in docker-executor jobs only.

Development

You need bun installed.

Scripts

# Install dependencies
bun install

# Run all tests
bun run test

# Run individual test-case
bunx vitest run tests/test-cases/cache-paths-not-array

It's also possible to run individual .gitlab-ci.yml, via bun src/index.ts --cwd examples/docker-compose-nodejs

Creating single executable binaries from source

bun install

# According to your needs:
bun build:linux-amd64
bun build:linux-arm64
bun build:win
bun build:macos-x64
bun build:macos-arm64
bun build-all
# the binary will be generated in the respective ./bin/<os>/gitlab-ci-local