Skip to main content

Ansible Secrets Management

Introduction​

When automating infrastructure with Ansible, you'll inevitably need to handle sensitive data such as passwords, API keys, SSL certificates, and other secrets. Storing these credentials in plain text within your playbooks or variable files poses significant security risks. This is where Ansible Vault comes in — a feature that allows you to encrypt sensitive data while still making it usable within your Ansible workflows.

In this guide, we'll explore how to effectively manage secrets in Ansible, focusing on Ansible Vault as the primary tool for encrypting sensitive information.

Why Secrets Management Matters​

Before diving into the mechanics, let's understand why proper secrets management is crucial:

  • Security Risk Mitigation: Prevents unauthorized access to sensitive credentials
  • Compliance Requirements: Helps meet regulatory standards that mandate protection of sensitive data
  • Collaboration: Enables secure sharing of Ansible code with team members
  • Version Control Safety: Allows you to store your infrastructure code in version control systems without exposing secrets

Getting Started with Ansible Vault​

Ansible Vault is included with Ansible by default, so there's no additional installation required. It uses AES256 encryption to protect your sensitive data.

Basic Vault Commands​

Here are the primary commands you'll use with Ansible Vault:

  • ansible-vault create: Create a new encrypted file
  • ansible-vault edit: Edit an encrypted file
  • ansible-vault view: View an encrypted file without editing
  • ansible-vault encrypt: Encrypt an existing file
  • ansible-vault decrypt: Decrypt an encrypted file
  • ansible-vault rekey: Change the encryption key on a vault-encrypted file

Let's explore each of these commands with examples.

Creating Encrypted Files​

To create a new encrypted file, use the ansible-vault create command:

bash
ansible-vault create secret_vars.yml

When you run this command, you'll be prompted to create a password. After entering and confirming your password, your default text editor will open, allowing you to add content to the file.

For example, you might add database credentials:

yaml
db_user: admin
db_password: super_secure_password123
db_host: database.example.com
api_key: 1a2b3c4d5e6f7g8h9i0j

Once you save and close the file, it will be encrypted. The encrypted content might look something like this:

$ANSIBLE_VAULT;1.1;AES256
63386438343933326463373435363232353262643533313763653433356663396130633330366639
3164333130633734346436643830376461613034616431630a643064356330613532363336643034
37663635333438313664383736383234373665623961326634663833623361323933326438666664
6230333535336334630a396565303237376161363137393932666665346537323131393130343334
3661

This encrypted format ensures your sensitive data is protected.

Editing Encrypted Files​

To edit an already encrypted file, use the ansible-vault edit command:

bash
ansible-vault edit secret_vars.yml

You'll be prompted for the password, and then your default editor will open with the decrypted content for editing.

Viewing Encrypted Files​

If you just want to view the content without editing:

bash
ansible-vault view secret_vars.yml

Again, you'll be prompted for the password, and the decrypted content will be displayed.

Encrypting Existing Files​

If you already have a file with sensitive data in plain text, you can encrypt it:

bash
ansible-vault encrypt existing_vars.yml

Decrypting Files​

To fully decrypt a file (converting it back to plain text):

bash
ansible-vault decrypt secret_vars.yml

Be cautious with this command as it removes the protection from your sensitive data.

Changing the Vault Password​

If you need to change the password used for encryption:

bash
ansible-vault rekey secret_vars.yml

You'll be asked for the current password and then for the new password.

Using Encrypted Files in Playbooks​

Once you have your secrets encrypted, you need a way to use them in your playbooks. There are several approaches:

Method 1: Providing the Password at Runtime​

You can run your playbook and provide the vault password as a command-line argument:

bash
ansible-playbook playbook.yml --ask-vault-pass

This will prompt you for the vault password before executing the playbook.

Here's what a playbook using encrypted variables might look like:

yaml
---
- name: Database Configuration
  hosts: database_servers
  vars_files:
    - secret_vars.yml  # This file is encrypted
  
  tasks:
    - name: Configure Database User
      mysql_user:
        name: "{{ db_user }}"
        password: "{{ db_password }}"
        host: "{{ db_host }}"
        priv: '*.*:ALL'
        state: present

Method 2: Using a Password File​

For automation purposes, you might want to avoid having to manually enter the password. You can create a password file:

bash
echo "your_vault_password" > .vault_pass
chmod 600 .vault_pass  # Restrict access to this file

Then, reference this file when running your playbook:

bash
ansible-playbook playbook.yml --vault-password-file=.vault_pass

You can also configure the password file in your ansible.cfg:

ini
[defaults]
vault_password_file = .vault_pass

With this configuration, you don't need to specify the password file on the command line.

Method 3: Using Multiple Vault Passwords​

In more complex environments, you might need different secrets for different environments (development, staging, production). Ansible supports using multiple vault IDs:

bash
ansible-vault encrypt --vault-id dev@dev_password.txt dev_secrets.yml
ansible-vault encrypt --vault-id prod@prod_password.txt prod_secrets.yml

Then when running your playbook:

bash
ansible-playbook playbook.yml --vault-id dev@dev_password.txt --vault-id prod@prod_password.txt

Best Practices for Ansible Vault​

To make the most of Ansible Vault while maintaining security, follow these best practices:

  1. Never commit unencrypted sensitive data: Ensure all secrets are encrypted before committing to version control.

  2. Use different vault passwords for different environments: Maintain separate passwords for development, staging, and production.

  3. Limit access to vault passwords: Only share vault passwords with team members who need access.

  4. Don't store vault passwords in version control: Keep your password files outside of your repository.

  5. Consider using a vault-id for better organization: This helps when you have multiple vault passwords.

  6. Only encrypt what needs to be encrypted: Encrypt only the sensitive values, not entire files when possible.

  7. Integrate with secret management tools: For larger environments, consider integrating with tools like HashiCorp Vault or AWS Secrets Manager.

Real-World Example: Multi-Environment Deployment​

Let's explore a practical example of managing secrets across multiple environments.

Directory Structure​

ansible/
├── ansible.cfg
├── playbooks/
│   └── deploy_app.yml
├── inventory/
│   ├── dev
│   └── prod
└── group_vars/
    ├── all/
    │   └── common_vars.yml
    ├── dev/
    │   └── vault.yml
    └── prod/
        └── vault.yml

Environment-Specific Encrypted Variables​

For dev environment (group_vars/dev/vault.yml):

yaml
---
db_user: dev_user
db_password: dev_password
api_key: dev_api_key

For production environment (group_vars/prod/vault.yml):

yaml
---
db_user: prod_user
db_password: very_secure_prod_password
api_key: prod_api_key

Encrypt these files with different passwords:

bash
ansible-vault encrypt group_vars/dev/vault.yml --vault-id dev@dev_password.txt
ansible-vault encrypt group_vars/prod/vault.yml --vault-id prod@prod_password.txt

Deployment Playbook​

yaml
---
- name: Deploy Application
  hosts: "{{ target_env }}"
  
  tasks:
    - name: Configure Database Connection
      template:
        src: templates/database.conf.j2
        dest: /etc/app/database.conf
      vars:
        db_connection_string: "mysql://{{ db_user }}:{{ db_password }}@{{ db_host }}/appdb"
      
    - name: Configure API Access
      template:
        src: templates/api.conf.j2
        dest: /etc/app/api.conf
      vars:
        api_auth: "Bearer {{ api_key }}"

Template File (templates/database.conf.j2)​

# Database Configuration
# Generated by Ansible - DO NOT EDIT MANUALLY

CONNECTION_STRING="{{ db_connection_string }}"

Running the Deployment​

For development:

bash
ansible-playbook playbooks/deploy_app.yml -i inventory/dev --vault-id dev@dev_password.txt -e "target_env=dev"

For production:

bash
ansible-playbook playbooks/deploy_app.yml -i inventory/prod --vault-id prod@prod_password.txt -e "target_env=prod"

Working with Partial Encryption​

Sometimes you don't want to encrypt an entire file, but just specific variables. You can use the ansible-vault encrypt_string command for this purpose:

bash
ansible-vault encrypt_string --vault-id dev@dev_password.txt 'supersecretvalue' --name 'api_key'

This will output something like:

yaml
api_key: !vault |
          $ANSIBLE_VAULT;1.2;AES256;dev
          63386438343933326463373435363232353262643533313763653433356663396130633330366639
          3164333130633734346436643830376461613034616431630a643064356330613532363336643034
          37663635333438313664383736383234373665623961326634663833623361323933326438666664
          6230333535336334630a396565303237376161363137393932666665346537323131393130343334
          3661

You can then paste this directly into your playbooks or variable files. This approach allows you to have both encrypted and non-encrypted variables in the same file.

Advanced Integration: Using Ansible Vault with External Secret Management​

For larger organizations, you might want to integrate Ansible with dedicated secret management solutions. Here's a brief overview of how to do that:

Integration with HashiCorp Vault​

Install the required packages:

bash
pip install hvac

Create a playbook that retrieves secrets from HashiCorp Vault:

yaml
---
- name: Retrieve secrets from HashiCorp Vault
  hosts: localhost
  vars:
    vault_addr: "https://vault.example.com:8200"
    vault_path: "secret/data/myapp"
  
  tasks:
    - name: Get secrets from HashiCorp Vault
      community.hashi_vault.vault_read:
        url: "{{ vault_addr }}"
        path: "{{ vault_path }}"
      register: vault_secrets
    
    - name: Use secrets in subsequent tasks
      debug:
        msg: "Retrieved database password: {{ vault_secrets.data.data.db_password }}"

Integration with AWS Secrets Manager​

Install the AWS collection:

bash
ansible-galaxy collection install amazon.aws

Create a playbook that retrieves secrets from AWS:

yaml
---
- name: Retrieve secrets from AWS Secrets Manager
  hosts: localhost
  
  tasks:
    - name: Get secret from AWS Secrets Manager
      amazon.aws.aws_secret:
        name: "my-app-secrets"
        region: "us-east-1"
      register: aws_secret
    
    - name: Use AWS secrets
      debug:
        msg: "The retrieved database user is {{ aws_secret.secret.db_user }}"

Troubleshooting Ansible Vault​

Here are solutions to common issues you might encounter:

"Decryption failed"​

This usually means you provided the wrong password. Double-check your password or password file.

bash
ansible-vault view --vault-id dev@dev_password.txt secret_vars.yml

"ERROR! Vault password file not found"​

Ensure your password file exists and is in the correct location.

bash
# Check if the file exists
ls -la .vault_pass

"ERROR! Attempting to decrypt but no vault secrets found"​

This typically means you're trying to decrypt a file that's not actually encrypted.

bash
# Check if the file is encrypted
head -1 secret_vars.yml
# It should start with "$ANSIBLE_VAULT;"

Summary​

Ansible Vault provides a robust way to manage secrets within your infrastructure automation:

  1. Encryption: Keep sensitive data secure using AES256 encryption
  2. Integration: Seamlessly use encrypted data in your playbooks
  3. Flexibility: Support for multiple environments with different secrets
  4. Collaboration: Enable team members to work with automation code without exposing sensitive data

By following the practices outlined in this guide, you can ensure that your Ansible automation is both powerful and secure, protecting sensitive information while maintaining operational efficiency.

Further Resources and Exercises​

Additional Resources​

Exercises to Practice​

  1. Basic Vault Usage:

    • Create an encrypted file containing database credentials
    • Create a playbook that uses these credentials
    • Run the playbook with --ask-vault-pass

  2. Multi-Environment Setup:

    • Create separate encrypted files for development and production environments
    • Set up a playbook that can deploy to either environment using the correct credentials

  3. Partial Encryption:

    • Create a vars file with a mix of encrypted and non-encrypted variables
    • Use this file in a playbook

  4. Integration Exercise:

    • If you have access to HashiCorp Vault or AWS Secrets Manager, try setting up integration with Ansible
    • Create a playbook that retrieves secrets from these external sources

By completing these exercises, you'll gain practical experience with Ansible's secrets management capabilities, preparing you for real-world automation scenarios.


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)