Skip to main content

Ansible Secrets Management

Introduction

When automating infrastructure with Ansible, you'll inevitably need to handle sensitive data such as passwords, API keys, SSL certificates, and other secrets. Storing these credentials in plain text within your playbooks or variable files poses significant security risks. This is where Ansible Vault comes in — a feature that allows you to encrypt sensitive data while still making it usable within your Ansible workflows.

In this guide, we'll explore how to effectively manage secrets in Ansible, focusing on Ansible Vault as the primary tool for encrypting sensitive information.

Why Secrets Management Matters

Before diving into the mechanics, let's understand why proper secrets management is crucial:

  • Security Risk Mitigation: Prevents unauthorized access to sensitive credentials
  • Compliance Requirements: Helps meet regulatory standards that mandate protection of sensitive data
  • Collaboration: Enables secure sharing of Ansible code with team members
  • Version Control Safety: Allows you to store your infrastructure code in version control systems without exposing secrets

Getting Started with Ansible Vault

Ansible Vault is included with Ansible by default, so there's no additional installation required. It uses AES256 encryption to protect your sensitive data.

Basic Vault Commands

Here are the primary commands you'll use with Ansible Vault:

  • ansible-vault create: Create a new encrypted file
  • ansible-vault edit: Edit an encrypted file
  • ansible-vault view: View an encrypted file without editing
  • ansible-vault encrypt: Encrypt an existing file
  • ansible-vault decrypt: Decrypt an encrypted file
  • ansible-vault rekey: Change the encryption key on a vault-encrypted file

Let's explore each of these commands with examples.

Creating Encrypted Files

To create a new encrypted file, use the ansible-vault create command:

bash
ansible-vault create secret_vars.yml

When you run this command, you'll be prompted to create a password. After entering and confirming your password, your default text editor will open, allowing you to add content to the file.

For example, you might add database credentials:

yaml
db_user: admin
db_password: super_secure_password123
db_host: database.example.com
api_key: 1a2b3c4d5e6f7g8h9i0j

Once you save and close the file, it will be encrypted. The encrypted content might look something like this:

$ANSIBLE_VAULT;1.1;AES256
63386438343933326463373435363232353262643533313763653433356663396130633330366639
3164333130633734346436643830376461613034616431630a643064356330613532363336643034
37663635333438313664383736383234373665623961326634663833623361323933326438666664
6230333535336334630a396565303237376161363137393932666665346537323131393130343334
3661

This encrypted format ensures your sensitive data is protected.

Editing Encrypted Files

To edit an already encrypted file, use the ansible-vault edit command:

bash
ansible-vault edit secret_vars.yml

You'll be prompted for the password, and then your default editor will open with the decrypted content for editing.

Viewing Encrypted Files

If you just want to view the content without editing:

bash
ansible-vault view secret_vars.yml

Again, you'll be prompted for the password, and the decrypted content will be displayed.

Encrypting Existing Files

If you already have a file with sensitive data in plain text, you can encrypt it:

bash
ansible-vault encrypt existing_vars.yml

Decrypting Files

To fully decrypt a file (converting it back to plain text):

bash
ansible-vault decrypt secret_vars.yml

Be cautious with this command as it removes the protection from your sensitive data.

Changing the Vault Password

If you need to change the password used for encryption:

bash
ansible-vault rekey secret_vars.yml

You'll be asked for the current password and then for the new password.

Using Encrypted Files in Playbooks

Once you have your secrets encrypted, you need a way to use them in your playbooks. There are several approaches:

Method 1: Providing the Password at Runtime

You can run your playbook and provide the vault password as a command-line argument:

bash
ansible-playbook playbook.yml --ask-vault-pass

This will prompt you for the vault password before executing the playbook.

Here's what a playbook using encrypted variables might look like:

yaml
---
- name: Database Configuration
hosts: database_servers
vars_files:
- secret_vars.yml # This file is encrypted

tasks:
- name: Configure Database User
mysql_user:
name: "{{ db_user }}"
password: "{{ db_password }}"
host: "{{ db_host }}"
priv: '*.*:ALL'
state: present

Method 2: Using a Password File

For automation purposes, you might want to avoid having to manually enter the password. You can create a password file:

bash
echo "your_vault_password" > .vault_pass
chmod 600 .vault_pass # Restrict access to this file

Then, reference this file when running your playbook:

bash
ansible-playbook playbook.yml --vault-password-file=.vault_pass

You can also configure the password file in your ansible.cfg:

ini
[defaults]
vault_password_file = .vault_pass

With this configuration, you don't need to specify the password file on the command line.

Method 3: Using Multiple Vault Passwords

In more complex environments, you might need different secrets for different environments (development, staging, production). Ansible supports using multiple vault IDs:

bash
ansible-vault encrypt --vault-id dev@dev_password.txt dev_secrets.yml
ansible-vault encrypt --vault-id prod@prod_password.txt prod_secrets.yml

Then when running your playbook:

bash
ansible-playbook playbook.yml --vault-id dev@dev_password.txt --vault-id prod@prod_password.txt

Best Practices for Ansible Vault

To make the most of Ansible Vault while maintaining security, follow these best practices:

  1. Never commit unencrypted sensitive data: Ensure all secrets are encrypted before committing to version control.

  2. Use different vault passwords for different environments: Maintain separate passwords for development, staging, and production.

  3. Limit access to vault passwords: Only share vault passwords with team members who need access.

  4. Don't store vault passwords in version control: Keep your password files outside of your repository.

  5. Consider using a vault-id for better organization: This helps when you have multiple vault passwords.

  6. Only encrypt what needs to be encrypted: Encrypt only the sensitive values, not entire files when possible.

  7. Integrate with secret management tools: For larger environments, consider integrating with tools like HashiCorp Vault or AWS Secrets Manager.

Real-World Example: Multi-Environment Deployment

Let's explore a practical example of managing secrets across multiple environments.

Directory Structure

ansible/
├── ansible.cfg
├── playbooks/
│ └── deploy_app.yml
├── inventory/
│ ├── dev
│ └── prod
└── group_vars/
├── all/
│ └── common_vars.yml
├── dev/
│ └── vault.yml
└── prod/
└── vault.yml

Environment-Specific Encrypted Variables

For dev environment (group_vars/dev/vault.yml):

yaml
---
db_user: dev_user
db_password: dev_password
api_key: dev_api_key

For production environment (group_vars/prod/vault.yml):

yaml
---
db_user: prod_user
db_password: very_secure_prod_password
api_key: prod_api_key

Encrypt these files with different passwords:

bash
ansible-vault encrypt group_vars/dev/vault.yml --vault-id dev@dev_password.txt
ansible-vault encrypt group_vars/prod/vault.yml --vault-id prod@prod_password.txt

Deployment Playbook

yaml
---
- name: Deploy Application
hosts: "{{ target_env }}"

tasks:
- name: Configure Database Connection
template:
src: templates/database.conf.j2
dest: /etc/app/database.conf
vars:
db_connection_string: "mysql://{{ db_user }}:{{ db_password }}@{{ db_host }}/appdb"

- name: Configure API Access
template:
src: templates/api.conf.j2
dest: /etc/app/api.conf
vars:
api_auth: "Bearer {{ api_key }}"

Template File (templates/database.conf.j2)

# Database Configuration
# Generated by Ansible - DO NOT EDIT MANUALLY

CONNECTION_STRING="{{ db_connection_string }}"

Running the Deployment

For development:

bash
ansible-playbook playbooks/deploy_app.yml -i inventory/dev --vault-id dev@dev_password.txt -e "target_env=dev"

For production:

bash
ansible-playbook playbooks/deploy_app.yml -i inventory/prod --vault-id prod@prod_password.txt -e "target_env=prod"

Working with Partial Encryption

Sometimes you don't want to encrypt an entire file, but just specific variables. You can use the ansible-vault encrypt_string command for this purpose:

bash
ansible-vault encrypt_string --vault-id dev@dev_password.txt 'supersecretvalue' --name 'api_key'

This will output something like:

yaml
api_key: !vault |
$ANSIBLE_VAULT;1.2;AES256;dev
63386438343933326463373435363232353262643533313763653433356663396130633330366639
3164333130633734346436643830376461613034616431630a643064356330613532363336643034
37663635333438313664383736383234373665623961326634663833623361323933326438666664
6230333535336334630a396565303237376161363137393932666665346537323131393130343334
3661

You can then paste this directly into your playbooks or variable files. This approach allows you to have both encrypted and non-encrypted variables in the same file.

Advanced Integration: Using Ansible Vault with External Secret Management

For larger organizations, you might want to integrate Ansible with dedicated secret management solutions. Here's a brief overview of how to do that:

Integration with HashiCorp Vault

Install the required packages:

bash
pip install hvac

Create a playbook that retrieves secrets from HashiCorp Vault:

yaml
---
- name: Retrieve secrets from HashiCorp Vault
hosts: localhost
vars:
vault_addr: "https://vault.example.com:8200"
vault_path: "secret/data/myapp"

tasks:
- name: Get secrets from HashiCorp Vault
community.hashi_vault.vault_read:
url: "{{ vault_addr }}"
path: "{{ vault_path }}"
register: vault_secrets

- name: Use secrets in subsequent tasks
debug:
msg: "Retrieved database password: {{ vault_secrets.data.data.db_password }}"

Integration with AWS Secrets Manager

Install the AWS collection:

bash
ansible-galaxy collection install amazon.aws

Create a playbook that retrieves secrets from AWS:

yaml
---
- name: Retrieve secrets from AWS Secrets Manager
hosts: localhost

tasks:
- name: Get secret from AWS Secrets Manager
amazon.aws.aws_secret:
name: "my-app-secrets"
region: "us-east-1"
register: aws_secret

- name: Use AWS secrets
debug:
msg: "The retrieved database user is {{ aws_secret.secret.db_user }}"

Troubleshooting Ansible Vault

Here are solutions to common issues you might encounter:

"Decryption failed"

This usually means you provided the wrong password. Double-check your password or password file.

bash
ansible-vault view --vault-id dev@dev_password.txt secret_vars.yml

"ERROR! Vault password file not found"

Ensure your password file exists and is in the correct location.

bash
# Check if the file exists
ls -la .vault_pass

"ERROR! Attempting to decrypt but no vault secrets found"

This typically means you're trying to decrypt a file that's not actually encrypted.

bash
# Check if the file is encrypted
head -1 secret_vars.yml
# It should start with "$ANSIBLE_VAULT;"

Summary

Ansible Vault provides a robust way to manage secrets within your infrastructure automation:

  1. Encryption: Keep sensitive data secure using AES256 encryption
  2. Integration: Seamlessly use encrypted data in your playbooks
  3. Flexibility: Support for multiple environments with different secrets
  4. Collaboration: Enable team members to work with automation code without exposing sensitive data

By following the practices outlined in this guide, you can ensure that your Ansible automation is both powerful and secure, protecting sensitive information while maintaining operational efficiency.

Further Resources and Exercises

Additional Resources

Exercises to Practice

  1. Basic Vault Usage:

    • Create an encrypted file containing database credentials
    • Create a playbook that uses these credentials
    • Run the playbook with --ask-vault-pass
  2. Multi-Environment Setup:

    • Create separate encrypted files for development and production environments
    • Set up a playbook that can deploy to either environment using the correct credentials
  3. Partial Encryption:

    • Create a vars file with a mix of encrypted and non-encrypted variables
    • Use this file in a playbook
  4. Integration Exercise:

    • If you have access to HashiCorp Vault or AWS Secrets Manager, try setting up integration with Ansible
    • Create a playbook that retrieves secrets from these external sources

By completing these exercises, you'll gain practical experience with Ansible's secrets management capabilities, preparing you for real-world automation scenarios.



If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)