CICD Ansible

Introduction

Ansible is a powerful open-source automation tool that can play a crucial role in Continuous Integration and Continuous Deployment (CI/CD) pipelines. Unlike traditional configuration management tools, Ansible is agentless, using SSH for communication with remote systems. When integrated into CI/CD workflows, Ansible enables automated infrastructure provisioning, configuration, and application deployment as part of your development pipeline.

In this guide, we'll explore how Ansible fits into the CI/CD ecosystem, how to set it up, and how to use it effectively to automate your infrastructure deployment alongside your application code.

What is Ansible?

Ansible is an IT automation platform that makes your applications and systems easier to deploy and maintain. It uses a simple, human-readable language called YAML to define automation tasks, making it accessible even to those new to automation.

Key features of Ansible include:

Agentless architecture: No need to install agents on managed nodes
Idempotent operations: Running the same playbook multiple times produces the same result
Declarative configuration: You specify the desired state, not the steps to get there
Extensible: Wide range of modules for various systems and services

Ansible in CI/CD Workflows

In a typical CI/CD pipeline, Ansible can handle the infrastructure and deployment stages. When developers push code changes, the CI server not only builds and tests the application but also triggers Ansible playbooks to provision and configure the target environment before deploying the application.

Setting Up Ansible for CI/CD

Prerequisites

Before integrating Ansible with your CI/CD pipeline, ensure you have:

A CI/CD server (Jenkins, GitLab CI, GitHub Actions, etc.)
SSH access to target servers
Ansible installed on your CI/CD server

Basic Setup

To install Ansible on your CI/CD server:

bash
# For Ubuntu/Debian
sudo apt update
sudo apt install ansible

# For CentOS/RHEL
sudo yum install ansible

# Verify installation
ansible --version

Output:

ansible [core 2.14.2]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0] (/usr/bin/python3)
  jinja version = 3.0.3
  libyaml = True

Project Structure

A typical project structure for Ansible in CI/CD might look like:

project-root/
├── .git/
├── src/                     # Application source code
├── tests/                   # Application tests
├── ansible/
│   ├── inventory/           # Host definitions
│   │   ├── production
│   │   └── staging
│   ├── group_vars/          # Variables for groups of hosts
│   ├── host_vars/           # Variables for specific hosts
│   ├── roles/               # Reusable automation components
│   │   ├── common/
│   │   ├── web-server/
│   │   └── database/
│   └── playbooks/           # Automation scripts
│       ├── provision.yml
│       └── deploy.yml
└── ci/                      # CI/CD configuration files
    ├── Jenkinsfile
    └── .gitlab-ci.yml

Creating Ansible Playbooks for CI/CD

Infrastructure Provisioning Playbook

Let's create a simple playbook to provision a web server environment:

yaml
# ansible/playbooks/provision.yml
---
- name: Provision Web Servers
  hosts: webservers
  become: true
  
  tasks:
    - name: Update apt cache
      apt:
        update_cache: yes
      when: ansible_os_family == "Debian"
      
    - name: Install required packages
      package:
        name:
          - nginx
          - python3
          - python3-pip
        state: present
        
    - name: Ensure nginx is running
      service:
        name: nginx
        state: started
        enabled: yes
        
    - name: Copy nginx configuration
      template:
        src: templates/nginx.conf.j2
        dest: /etc/nginx/sites-available/default
      notify: Restart nginx
      
  handlers:
    - name: Restart nginx
      service:
        name: nginx
        state: restarted

Application Deployment Playbook

Now, let's create a playbook to deploy our application:

yaml
# ansible/playbooks/deploy.yml
---
- name: Deploy Application
  hosts: webservers
  become: true
  
  vars:
    app_dir: /var/www/myapp
    artifact_path: "{{ lookup('env', 'CI_ARTIFACT_PATH') }}"
    
  tasks:
    - name: Ensure application directory exists
      file:
        path: "{{ app_dir }}"
        state: directory
        owner: www-data
        group: www-data
        mode: '0755'
        
    - name: Copy application files
      unarchive:
        src: "{{ artifact_path }}"
        dest: "{{ app_dir }}"
        owner: www-data
        group: www-data
        remote_src: no
        
    - name: Restart application service
      service:
        name: myapp
        state: restarted
      
    - name: Verify deployment
      uri:
        url: http://localhost/health
        status_code: 200
      register: result
      retries: 5
      delay: 10
      until: result.status == 200

Integrating Ansible with CI/CD Tools

Jenkins Integration

Here's how to integrate Ansible with Jenkins using a Jenkinsfile:

groovy
pipeline {
    agent any
    
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        
        stage('Build') {
            steps {
                sh 'npm install'
                sh 'npm run build'
                sh 'tar -czf app.tar.gz dist/'
            }
        }
        
        stage('Test') {
            steps {
                sh 'npm test'
            }
        }
        
        stage('Provision Infrastructure') {
            when {
                branch 'main'
            }
            steps {
                withCredentials([sshUserPrivateKey(credentialsId: 'ansible-ssh-key', keyFileVariable: 'SSH_KEY')]) {
                    sh '''
                    export ANSIBLE_HOST_KEY_CHECKING=False
                    ansible-playbook -i ansible/inventory/production ansible/playbooks/provision.yml --private-key=$SSH_KEY
                    '''
                }
            }
        }
        
        stage('Deploy Application') {
            when {
                branch 'main'
            }
            steps {
                withCredentials([sshUserPrivateKey(credentialsId: 'ansible-ssh-key', keyFileVariable: 'SSH_KEY')]) {
                    sh '''
                    export ANSIBLE_HOST_KEY_CHECKING=False
                    export CI_ARTIFACT_PATH=app.tar.gz
                    ansible-playbook -i ansible/inventory/production ansible/playbooks/deploy.yml --private-key=$SSH_KEY
                    '''
                }
            }
        }
    }
    
    post {
        always {
            cleanWs()
        }
    }
}

GitLab CI Integration

Here's how to integrate Ansible with GitLab CI using .gitlab-ci.yml:

yaml
stages:
  - build
  - test
  - provision
  - deploy

build:
  stage: build
  script:
    - npm install
    - npm run build
    - tar -czf app.tar.gz dist/
  artifacts:
    paths:
      - app.tar.gz

test:
  stage: test
  script:
    - npm test

provision:
  stage: provision
  script:
    - apt-get update -qq && apt-get install -y ansible
    - mkdir -p ~/.ssh
    - echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
    - chmod 600 ~/.ssh/id_rsa
    - export ANSIBLE_HOST_KEY_CHECKING=False
    - ansible-playbook -i ansible/inventory/production ansible/playbooks/provision.yml
  only:
    - main

deploy:
  stage: deploy
  script:
    - apt-get update -qq && apt-get install -y ansible
    - mkdir -p ~/.ssh
    - echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
    - chmod 600 ~/.ssh/id_rsa
    - export ANSIBLE_HOST_KEY_CHECKING=False
    - export CI_ARTIFACT_PATH=app.tar.gz
    - ansible-playbook -i ansible/inventory/production ansible/playbooks/deploy.yml
  only:
    - main

GitHub Actions Integration

Here's how to integrate Ansible with GitHub Actions:

yaml
name: CI/CD Pipeline with Ansible

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '16'
        
    - name: Install dependencies
      run: npm install
      
    - name: Build
      run: npm run build
      
    - name: Test
      run: npm test
      
    - name: Archive application artifact
      if: github.ref == 'refs/heads/main'
      run: tar -czf app.tar.gz dist/
      
    - name: Upload artifact
      if: github.ref == 'refs/heads/main'
      uses: actions/upload-artifact@v3
      with:
        name: app-artifact
        path: app.tar.gz
        
  deploy:
    if: github.ref == 'refs/heads/main'
    needs: build-and-test
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Download artifact
      uses: actions/download-artifact@v3
      with:
        name: app-artifact
        
    - name: Set up Ansible
      run: |
        sudo apt-add-repository --yes --update ppa:ansible/ansible
        sudo apt install ansible
        
    - name: Set up SSH
      run: |
        mkdir -p ~/.ssh
        echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
        chmod 600 ~/.ssh/id_rsa
        ssh-keyscan -H ${{ secrets.SERVER_IP }} >> ~/.ssh/known_hosts
        
    - name: Provision infrastructure
      run: |
        export ANSIBLE_HOST_KEY_CHECKING=False
        ansible-playbook -i ansible/inventory/production ansible/playbooks/provision.yml
        
    - name: Deploy application
      run: |
        export ANSIBLE_HOST_KEY_CHECKING=False
        export CI_ARTIFACT_PATH=app.tar.gz
        ansible-playbook -i ansible/inventory/production ansible/playbooks/deploy.yml

Best Practices

1. Use Ansible Vault for Secrets

Sensitive information like passwords and API keys should be encrypted using Ansible Vault:

bash
# Create an encrypted file
ansible-vault create secrets.yml

# Edit an encrypted file
ansible-vault edit secrets.yml

# Use the vault file in your playbook
ansible-playbook playbook.yml --ask-vault-pass

In CI/CD pipelines, you can pass the vault password as an environment variable:

bash
ansible-playbook playbook.yml --vault-password-file=vault-password-file

2. Use Roles for Reusable Components

Structure your Ansible code using roles to make it more maintainable and reusable:

yaml
# ansible/playbooks/site.yml
---
- hosts: webservers
  become: true
  roles:
    - common
    - web-server
    
- hosts: databases
  become: true
  roles:
    - common
    - database

3. Test Your Playbooks

Use Ansible Molecule for testing your roles:

yaml
# Install Molecule
pip install molecule[docker]

# Initialize a new role with testing
molecule init role my-role

# Test your role
cd my-role
molecule test

4. Version Control Your Ansible Code

Keep your Ansible code in version control alongside your application code to ensure both evolve together.

5. Use Dynamic Inventories

For cloud environments, use dynamic inventories to automatically discover and manage resources:

bash
# For AWS
pip install boto3
ansible-playbook -i inventory/aws_ec2.yml playbook.yml

Practical Example: Full CI/CD Pipeline with Ansible

Let's build a complete example for deploying a Node.js application through a CI/CD pipeline using Ansible.

1. Application Structure

nodejs-app/
├── src/
│   └── app.js
├── package.json
├── tests/
│   └── app.test.js
└── ansible/
    ├── inventory/
    │   └── production
    └── playbooks/
        ├── provision.yml
        └── deploy.yml

2. Sample Node.js Application

javascript
// app.js
const express = require('express');
const app = express();
const port = process.env.PORT || 3000;

app.get('/', (req, res) => {
  res.send('Hello from CI/CD deployed with Ansible!');
});

app.get('/health', (req, res) => {
  res.status(200).send('OK');
});

app.listen(port, () => {
  console.log(`App listening at http://localhost:${port}`);
});

module.exports = app;

3. Comprehensive Ansible Deployment

Our updated deployment playbook:

yaml
# ansible/playbooks/deploy.yml
---
- name: Deploy Node.js Application
  hosts: webservers
  become: true
  
  vars:
    app_name: nodejs-app
    app_user: nodejs
    app_dir: /opt/{{ app_name }}
    artifact_path: "{{ lookup('env', 'CI_ARTIFACT_PATH') }}"
    node_version: "16.x"
    
  tasks:
    - name: Setup Node.js repository
      shell: |
        curl -fsSL https://deb.nodesource.com/setup_{{ node_version }} | bash -
      args:
        warn: false
      when: ansible_os_family == "Debian"
        
    - name: Install Node.js and npm
      package:
        name: nodejs
        state: present
        
    - name: Ensure app user exists
      user:
        name: "{{ app_user }}"
        state: present
        system: yes
        
    - name: Ensure application directory exists
      file:
        path: "{{ app_dir }}"
        state: directory
        owner: "{{ app_user }}"
        group: "{{ app_user }}"
        mode: '0755'
        
    - name: Extract application archive
      unarchive:
        src: "{{ artifact_path }}"
        dest: "{{ app_dir }}"
        owner: "{{ app_user }}"
        group: "{{ app_user }}"
        remote_src: no
        
    - name: Install dependencies
      npm:
        path: "{{ app_dir }}"
        state: present
        production: yes
      become_user: "{{ app_user }}"
        
    - name: Create systemd service file
      template:
        src: templates/nodejs-app.service.j2
        dest: /etc/systemd/system/{{ app_name }}.service
      notify: Reload systemd
        
    - name: Configure Nginx as reverse proxy
      template:
        src: templates/nginx-app.conf.j2
        dest: /etc/nginx/sites-available/{{ app_name }}
      notify: Reload nginx
        
    - name: Enable Nginx site
      file:
        src: /etc/nginx/sites-available/{{ app_name }}
        dest: /etc/nginx/sites-enabled/{{ app_name }}
        state: link
      notify: Reload nginx
        
    - name: Ensure services are running
      service:
        name: "{{ item }}"
        state: started
        enabled: yes
      loop:
        - "{{ app_name }}"
        - nginx
        
    - name: Wait for application to be ready
      uri:
        url: http://localhost/health
        status_code: 200
      register: result
      retries: 6
      delay: 10
      until: result.status == 200
        
  handlers:
    - name: Reload systemd
      systemd:
        daemon_reload: yes
        
    - name: Reload nginx
      service:
        name: nginx
        state: reloaded

4. Service Template File

# templates/nodejs-app.service.j2
[Unit]
Description={{ app_name }} service
After=network.target

[Service]
Environment=NODE_ENV=production
Environment=PORT=3000
Type=simple
User={{ app_user }}
WorkingDirectory={{ app_dir }}
ExecStart=/usr/bin/node {{ app_dir }}/src/app.js
Restart=on-failure

[Install]
WantedBy=multi-user.target

5. Nginx Configuration Template

# templates/nginx-app.conf.j2
server {
    listen 80;
    server_name {{ ansible_host }};

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

6. CI/CD Configuration with GitHub Actions

yaml
name: Node.js CI/CD with Ansible

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '16'
        
    - name: Install dependencies
      run: npm install
      
    - name: Run tests
      run: npm test
      
    - name: Archive application
      if: github.ref == 'refs/heads/main'
      run: |
        tar -czf app.tar.gz src/ package.json package-lock.json
      
    - name: Upload artifact
      if: github.ref == 'refs/heads/main'
      uses: actions/upload-artifact@v3
      with:
        name: app-artifact
        path: app.tar.gz
        
  deploy:
    if: github.ref == 'refs/heads/main'
    needs: build-and-test
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Download artifact
      uses: actions/download-artifact@v3
      with:
        name: app-artifact
        
    - name: Install Ansible
      run: |
        sudo apt-add-repository --yes --update ppa:ansible/ansible
        sudo apt install ansible
        
    - name: Set up SSH
      run: |
        mkdir -p ~/.ssh
        echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
        chmod 600 ~/.ssh/id_rsa
        ssh-keyscan -H ${{ secrets.PRODUCTION_SERVER }} >> ~/.ssh/known_hosts
        
    - name: Deploy with Ansible
      run: |
        export ANSIBLE_HOST_KEY_CHECKING=False
        export CI_ARTIFACT_PATH=app.tar.gz
        ansible-playbook -i ansible/inventory/production ansible/playbooks/deploy.yml

Troubleshooting Common Issues

1. SSH Connection Issues

If Ansible cannot connect to your servers:

bash
# Disable host key checking
export ANSIBLE_HOST_KEY_CHECKING=False

# Specify SSH key explicitly
ansible-playbook -i inventory playbook.yml --private-key=~/.ssh/id_rsa

2. Permissions Problems

If you encounter permission issues during deployment:

yaml
# Ensure proper ownership and permissions
- name: Set file ownership
  file:
    path: "{{ app_dir }}"
    state: directory
    recurse: yes
    owner: "{{ app_user }}"
    group: "{{ app_user }}"

3. Debugging Playbooks

To troubleshoot playbook execution:

bash
# Run in verbose mode
ansible-playbook -i inventory playbook.yml -vvv

# Run with step mode to confirm each task
ansible-playbook -i inventory playbook.yml --step

Summary

Integrating Ansible into your CI/CD pipeline provides powerful infrastructure automation capabilities. By combining application deployment with infrastructure provisioning, you create a complete end-to-end deployment process that ensures consistency across environments.

Key takeaways:

Ansible is an agentless automation tool that fits perfectly into CI/CD workflows
Playbooks define your infrastructure and deployment as code
Integration with CI/CD tools like Jenkins, GitLab CI, and GitHub Actions is straightforward
Using best practices like Ansible Vault for secrets, roles for organization, and testing ensures reliable deployments
A well-designed Ansible CI/CD pipeline can significantly reduce deployment time and human error

Additional Resources

Ansible Documentation
Ansible Galaxy - Repository of Ansible roles
Ansible for DevOps - Book by Jeff Geerling
Ansible GitHub Repository

Exercises

Create a simple Ansible playbook that installs and configures Nginx on a target server.
Extend the playbook to deploy a static website from a Git repository.
Create a CI/CD pipeline using GitHub Actions that uses Ansible to deploy a simple application.
Implement environment-specific configurations using Ansible inventory variables.
Create an Ansible role for a database server and integrate it into your deployment pipeline.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)

Introduction​

What is Ansible?​

Ansible in CI/CD Workflows​

Setting Up Ansible for CI/CD​

Prerequisites​

Basic Setup​

Project Structure​

Creating Ansible Playbooks for CI/CD​

Infrastructure Provisioning Playbook​

Application Deployment Playbook​

Integrating Ansible with CI/CD Tools​

Jenkins Integration​

GitLab CI Integration​

GitHub Actions Integration​

Best Practices​

1. Use Ansible Vault for Secrets​

2. Use Roles for Reusable Components​

3. Test Your Playbooks​

4. Version Control Your Ansible Code​

5. Use Dynamic Inventories​

Practical Example: Full CI/CD Pipeline with Ansible​

1. Application Structure​

2. Sample Node.js Application​

3. Comprehensive Ansible Deployment​

4. Service Template File​

5. Nginx Configuration Template​

6. CI/CD Configuration with GitHub Actions​

Troubleshooting Common Issues​

1. SSH Connection Issues​

2. Permissions Problems​

3. Debugging Playbooks​

Summary​

Additional Resources​

Exercises​