Skip to main content

GitLab CI/CD

Introduction​

Continuous Integration and Continuous Deployment (CI/CD) is a fundamental practice in modern software development that automates the building, testing, and deployment of applications. GitLab offers a powerful built-in CI/CD system that helps developers maintain code quality and deliver software faster.

In this tutorial, we'll explore how GitLab CI/CD works, set up a basic pipeline, and learn best practices for implementing CI/CD in your projects.

What is GitLab CI/CD?​

GitLab CI/CD is an integrated tool within the GitLab platform that allows you to:

  • Automatically build your application with each change
  • Run tests to ensure code quality
  • Deploy your application to various environments
  • Automate repetitive tasks in your development workflow

The core concept of GitLab CI/CD revolves around pipelines - a series of automated steps that your code goes through from development to production.

Key Concepts​

Before diving into implementation, let's understand some key concepts:

Pipelines​

A pipeline is a group of jobs that get executed in stages. Each stage can have multiple jobs that run in parallel.

Jobs​

Jobs are the most basic building blocks of a GitLab CI/CD pipeline. A job defines:

  • What to execute (the script)
  • When to execute it (the stage)
  • Where to execute it (the runner)

Stages​

Stages group similar jobs together and define the order of execution. For example, a typical pipeline might have these stages:

  1. Build
  2. Test
  3. Deploy

Runners​

Runners are agents that run your CI/CD jobs. GitLab provides shared runners, or you can set up your own specific runners.

Setting Up Your First CI/CD Pipeline​

The .gitlab-ci.yml File​

GitLab CI/CD is configured through a file called .gitlab-ci.yml placed in the root of your repository. This file defines what jobs to run, when to run them, and what to do when they succeed or fail.

Let's create a basic .gitlab-ci.yml file for a Node.js application:

yaml
# Define stages
stages:
  - build
  - test
  - deploy

# Cache node modules between jobs
cache:
  paths:
    - node_modules/

# Build job
build:
  stage: build
  image: node:16
  script:
    - npm install
    - npm run build
  artifacts:
    paths:
      - dist/

# Test job
test:
  stage: test
  image: node:16
  script:
    - npm install
    - npm run test

# Deploy job
deploy:
  stage: deploy
  image: node:16
  script:
    - echo "Deploying application..."
    - npm install -g netlify-cli
    - netlify deploy --prod
  only:
    - main  # Only run this job on the main branch

This basic configuration:

  1. Defines three stages: build, test, and deploy
  2. Sets up caching for node_modules to speed up subsequent pipeline runs
  3. Configures a build job that installs dependencies and builds the application
  4. Sets up a test job that runs the test suite
  5. Creates a deploy job that deploys to Netlify (only on the main branch)

Running Your Pipeline​

Once you've added the .gitlab-ci.yml file to your repository and pushed it, GitLab will automatically detect and run your pipeline. You can view the status and logs of your pipeline in the CI/CD section of your GitLab project.

Advanced GitLab CI/CD Features​

Environment Variables​

You can define environment variables for your jobs, either in the GitLab UI or in your .gitlab-ci.yml file:

yaml
variables:
  NODE_ENV: "production"

deploy:
  stage: deploy
  script:
    - echo "Deploying with NODE_ENV=$NODE_ENV"
  variables:
    API_TOKEN: $API_TOKEN  # Uses a predefined CI/CD variable

For sensitive information like API tokens, you should use GitLab's CI/CD variables feature in your project settings.

Conditional Jobs​

You can make jobs run only under certain conditions:

yaml
deploy_staging:
  stage: deploy
  script:
    - echo "Deploying to staging..."
  only:
    - develop

deploy_production:
  stage: deploy
  script:
    - echo "Deploying to production..."
  only:
    - main
  when: manual  # Requires manual intervention to run

Parallelizing Tests​

To speed up testing, you can parallelize your test suite:

yaml
test:
  stage: test
  parallel: 3
  script:
    - npm install
    - npm run test:ci -- --split=$CI_NODE_INDEX/$CI_NODE_TOTAL

Multi-Stage Deployments​

You can implement progressive deployments with environments:

yaml
deploy_staging:
  stage: deploy
  script:
    - echo "Deploying to staging..."
  environment:
    name: staging
    url: https://staging.example.com
  only:
    - develop

deploy_production:
  stage: deploy
  script:
    - echo "Deploying to production..."
  environment:
    name: production
    url: https://example.com
  only:
    - main
  when: manual

Real-World Example: Node.js Application​

Let's look at a more comprehensive example for a Node.js application with frontend and backend components:

yaml
stages:
  - install
  - lint
  - test
  - build
  - deploy

variables:
  NODE_ENV: development

# Install dependencies
install:
  stage: install
  image: node:16
  script:
    - npm ci
  cache:
    paths:
      - node_modules/
    policy: push

# Lint code
lint:
  stage: lint
  image: node:16
  script:
    - npm ci
    - npm run lint
  cache:
    paths:
      - node_modules/
    policy: pull
  dependencies:
    - install

# Run tests
test:
  stage: test
  image: node:16
  script:
    - npm ci
    - npm run test:coverage
  coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
  artifacts:
    paths:
      - coverage/
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
  cache:
    paths:
      - node_modules/
    policy: pull
  dependencies:
    - install

# Build application
build:
  stage: build
  image: node:16
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - build/
  cache:
    paths:
      - node_modules/
    policy: pull
  dependencies:
    - install
  only:
    - main
    - develop

# Deploy to staging
deploy_staging:
  stage: deploy
  image: node:16
  script:
    - npm install -g firebase-tools
    - firebase use staging
    - firebase deploy --only hosting -m "Pipeline $CI_PIPELINE_ID, commit $CI_COMMIT_SHA"
  environment:
    name: staging
    url: https://staging.myapp.com
  only:
    - develop
  when: on_success
  dependencies:
    - build

# Deploy to production
deploy_production:
  stage: deploy
  image: node:16
  script:
    - npm install -g firebase-tools
    - firebase use production
    - firebase deploy --only hosting -m "Pipeline $CI_PIPELINE_ID, commit $CI_COMMIT_SHA"
  environment:
    name: production
    url: https://myapp.com
  only:
    - main
  when: manual
  dependencies:
    - build

This real-world pipeline:

  1. Installs dependencies once and caches them
  2. Runs linting to ensure code quality
  3. Executes tests and collects coverage reports
  4. Builds the application for deployment
  5. Deploys to staging automatically when code is merged to the develop branch
  6. Deploys to production manually when approved (after merging to main)

Best Practices for GitLab CI/CD​

1. Keep Pipelines Fast​

  • Use caching effectively
  • Parallelize jobs when possible
  • Only run necessary jobs (use only and except)

2. Secure Your Pipelines​

  • Use CI/CD variables for sensitive information
  • Implement review environments for pull requests
  • Set proper permissions for protected branches

3. Structure Your Pipelines Clearly​

  • Group related jobs in stages
  • Use meaningful names for jobs and stages
  • Document complex scripts

4. Use Templates and Includes​

For large projects, you can split your CI configuration using includes:

yaml
include:
  - local: 'ci/build.yml'
  - local: 'ci/test.yml'
  - local: 'ci/deploy.yml'

stages:
  - build
  - test
  - deploy

5. Monitor and Optimize​

  • Review pipeline metrics
  • Set up notifications for failures
  • Regularly refine your CI/CD process

Troubleshooting Common Issues​

Pipeline Not Running​

If your pipeline isn't running, check:

  • Is the .gitlab-ci.yml file valid? (Check syntax)
  • Are runners available for your project?
  • Are you using the correct branch names in your conditions?

Jobs Failing​

For failing jobs:

  • Examine the job logs carefully
  • Try running commands locally first
  • Use the CI/CD linter in GitLab to validate your configuration

Slow Pipelines​

If your pipelines are slow:

  • Review caching strategy
  • Only keep necessary artifacts
  • Consider upgrading runners or adding more parallel runners

Summary​

GitLab CI/CD provides a powerful, integrated solution for automating your software development workflow. By following the principles and examples in this guide, you can:

  • Automate building, testing, and deploying your applications
  • Maintain high code quality through automated testing
  • Implement progressive delivery with staging and production environments
  • Speed up your development process

Remember that CI/CD is not just about tools, but about adopting a culture of automation and continuous improvement in your development process.

Additional Resources​

Exercises​

  1. Basic Pipeline: Create a simple .gitlab-ci.yml file for your project that builds and tests your code.

  2. Environment Variables: Add environment-specific variables to your pipeline for different deployment targets.

  3. Advanced Pipeline: Implement a multi-stage pipeline with staging and production deployments, with production requiring manual approval.

  4. Optimizations: Review an existing pipeline and identify ways to make it faster using caching and parallelization.


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)