Ansible Cloud Modules
Introduction
Ansible Cloud Modules provide a powerful way to automate and manage resources across various cloud providers. These modules extend Ansible's core functionality, allowing you to provision, configure, and orchestrate cloud infrastructure using the same declarative approach you use for managing servers.
In this guide, we'll explore how Ansible Cloud Modules work, their key features, and how to use them to automate infrastructure across popular cloud platforms like AWS, Azure, Google Cloud, and more.
Understanding Ansible Cloud Modules
Ansible Cloud Modules are specialized components that interact with cloud provider APIs, enabling you to manage cloud resources as part of your automation workflows.
Key Benefits
- Unified Syntax: Use the same YAML-based syntax for all cloud providers
- Idempotent Operations: Apply changes only when needed
- Infrastructure as Code: Define your cloud infrastructure as version-controlled code
- No Agents Required: Works with cloud provider APIs directly without requiring agents
- Cross-Platform: Support for multiple cloud providers with consistent patterns
Types of Cloud Modules
Ansible offers modules for all major cloud providers:
Getting Started with Cloud Modules
Before using Ansible Cloud Modules, you need to install the required collections and set up authentication for your cloud provider.
Installing Cloud Collections
Ansible moved cloud modules to dedicated collections in Ansible 2.10. Install them using:
# AWS Collection
ansible-galaxy collection install amazon.aws
# Azure Collection
ansible-galaxy collection install azure.azcollection
# Google Cloud Collection
ansible-galaxy collection install google.cloud
Setting Up Authentication
Each cloud provider requires specific authentication methods. Here are some common approaches:
AWS Authentication
# Via environment variables
- name: Set environment variables
hosts: localhost
vars:
ansible_python_interpreter: /usr/bin/python3
tasks:
- name: Set AWS credentials
ansible.builtin.shell: |
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_REGION=us-west-2
Alternatively, you can provide credentials directly in your playbook (though environment variables or credential files are preferred for security):
- name: Create EC2 instance
hosts: localhost
tasks:
- name: Launch instance
amazon.aws.ec2_instance:
aws_access_key: AKIAIOSFODNN7EXAMPLE
aws_secret_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
region: us-west-2
image_id: ami-123456
instance_type: t2.micro
wait: yes
vpc_subnet_id: subnet-29e63245
tags:
Name: ansible-created-instance
Practical Examples
Let's look at some examples of using Ansible Cloud Modules with different providers.
AWS Examples
Creating EC2 Instances
- name: Provision EC2 instances
hosts: localhost
collections:
- amazon.aws
tasks:
- name: Launch EC2 instance
ec2_instance:
name: "web-server"
key_name: "my-key"
vpc_subnet_id: subnet-123456
instance_type: t2.micro
security_group: default
network:
assign_public_ip: yes
image_id: ami-0c55b159cbfafe1f0
tags:
Environment: Production
Role: WebServer
wait: yes
count: 3
register: ec2
- name: Add instances to inventory
add_host:
name: "{{ item.public_ip_address }}"
groups: webservers
ansible_host: "{{ item.public_ip_address }}"
ansible_user: ec2-user
loop: "{{ ec2.instances }}"
Managing S3 Buckets
- name: Manage S3 buckets
hosts: localhost
collections:
- amazon.aws
tasks:
- name: Create an S3 bucket
s3_bucket:
name: my-ansible-bucket-2023
region: us-east-1
state: present
tags:
Environment: Development
Project: Ansible-Demo
- name: Upload file to S3 bucket
aws_s3:
bucket: my-ansible-bucket-2023
object: /path/in/bucket/file.txt
src: /local/path/to/file.txt
mode: put
Azure Examples
Creating a Resource Group
- name: Create Azure Resource Group
hosts: localhost
collections:
- azure.azcollection
tasks:
- name: Create resource group
azure_rm_resourcegroup:
name: ansible-rg
location: eastus
tags:
Environment: Development
Project: AnsibleAutomation
Deploying a Virtual Machine
- name: Create Azure VM
hosts: localhost
collections:
- azure.azcollection
tasks:
- name: Create Azure VM
azure_rm_virtualmachine:
resource_group: ansible-rg
name: ansible-vm
vm_size: Standard_B1s
admin_username: azureuser
ssh_password_enabled: false
ssh_public_keys:
- path: /home/azureuser/.ssh/authorized_keys
key_data: "ssh-rsa AAA...zzz"
image:
offer: UbuntuServer
publisher: Canonical
sku: 18.04-LTS
version: latest
network_interfaces: my-nic
register: azure_vm
- name: Get VM facts
azure_rm_virtualmachine_info:
resource_group: ansible-rg
name: ansible-vm
register: vm_facts
- name: Display public IP
debug:
msg: "VM public IP: {{ vm_facts.virtualmachines[0].properties.networkProfile.networkInterfaces[0].properties.ipConfigurations[0].properties.publicIPAddress.properties.ipAddress }}"
Google Cloud Examples
Creating a GCP Instance
- name: Create GCP VM instance
hosts: localhost
collections:
- google.cloud
tasks:
- name: Create a VM instance
google.cloud.gcp_compute_instance:
name: ansible-instance
machine_type: e2-medium
disks:
- auto_delete: true
boot: true
initialize_params:
source_image: projects/debian-cloud/global/images/family/debian-10
network_interfaces:
- network: default
access_configs:
- name: External NAT
nat_ip: "{{ gcp_address }}"
type: ONE_TO_ONE_NAT
zone: us-central1-a
project: my-project
auth_kind: serviceaccount
service_account_file: /path/to/service-account.json
state: present
register: gcp_vm
Advanced Usage: Cloud Orchestration
Ansible Cloud Modules really shine when orchestrating complex deployments that span multiple cloud resources.
Multi-Tier Application Deployment
Here's an example deploying a multi-tier application on AWS:
- name: Deploy Multi-Tier Application
hosts: localhost
collections:
- amazon.aws
vars:
vpc_id: vpc-12345678
web_subnet_id: subnet-abcdef12
db_subnet_id: subnet-12345678
app_name: my-web-app
region: us-east-1
tasks:
# Create security groups
- name: Create web security group
ec2_security_group:
name: "{{ app_name }}-web-sg"
description: Security group for web servers
vpc_id: "{{ vpc_id }}"
region: "{{ region }}"
rules:
- proto: tcp
ports: 80
cidr_ip: 0.0.0.0/0
- proto: tcp
ports: 443
cidr_ip: 0.0.0.0/0
- proto: tcp
ports: 22
cidr_ip: 10.0.0.0/8
register: web_sg
- name: Create database security group
ec2_security_group:
name: "{{ app_name }}-db-sg"
description: Security group for database servers
vpc_id: "{{ vpc_id }}"
region: "{{ region }}"
rules:
- proto: tcp
ports: 3306
group_id: "{{ web_sg.group_id }}"
register: db_sg
# Launch database tier
- name: Launch RDS instance
rds_instance:
id: "{{ app_name }}-db"
state: present
engine: mysql
engine_version: 5.7
db_instance_class: db.t3.micro
allocated_storage: 20
storage_type: gp2
username: dbadmin
password: "{{ db_password }}"
vpc_security_group_ids:
- "{{ db_sg.group_id }}"
db_subnet_group_name: my-db-subnet-group
tags:
Application: "{{ app_name }}"
Environment: Production
register: rds
# Launch web tier
- name: Create web servers
ec2_instance:
name: "{{ app_name }}-web"
image_id: ami-0c55b159cbfafe1f0
instance_type: t2.micro
security_group: "{{ web_sg.group_id }}"
vpc_subnet_id: "{{ web_subnet_id }}"
count: 2
user_data: |
#!/bin/bash
yum update -y
yum install -y httpd php php-mysql
echo "<?php phpinfo(); ?>" > /var/www/html/info.php
systemctl start httpd
systemctl enable httpd
tags:
Name: "{{ app_name }}-web"
Application: "{{ app_name }}"
Environment: Production
register: web_instances
# Create load balancer
- name: Create application load balancer
elb_application_lb:
name: "{{ app_name }}-alb"
security_groups:
- "{{ web_sg.group_id }}"
subnets:
- "{{ web_subnet_id }}"
listeners:
- Protocol: HTTP
Port: 80
DefaultActions:
- Type: forward
TargetGroupName: "{{ app_name }}-tg"
state: present
register: alb
- name: Create target group
elb_target_group:
name: "{{ app_name }}-tg"
protocol: HTTP
port: 80
vpc_id: "{{ vpc_id }}"
health_check_protocol: HTTP
health_check_path: /info.php
successful_response_codes: "200"
target_type: instance
targets:
- Id: "{{ item.instance_id }}"
Port: 80
state: present
loop: "{{ web_instances.instances }}"
Cloud Inventory Integration
Ansible can dynamically discover cloud resources, allowing you to run playbooks against your existing cloud infrastructure.
Dynamic Inventory Example for AWS
Create a file named aws_ec2.yml with the following content:
plugin: aws_ec2
regions:
- us-east-1
- us-west-2
keyed_groups:
- key: tags.Environment
prefix: env
- key: tags.Role
prefix: role
groups:
development: "'Development' in tags.Environment"
webservers: "'WebServer' in tags.Role"
Then run your playbook using this inventory:
ansible-playbook -i aws_ec2.yml playbook.yml
This will discover all EC2 instances in the specified regions and group them according to their tags.
Best Practices for Cloud Automation
When working with Ansible Cloud Modules, follow these best practices:
-
Use Collections: Always use the latest collections for the most up-to-date modules.
-
Secure Credentials: Never hardcode credentials in playbooks. Use environment variables, vault, or provider-specific credential files.
-
Implement Tagging: Tag all resources consistently to make management and inventory organization easier.
-
Test Idempotence: Ensure your playbooks are truly idempotent by running them multiple times.
-
Use Wait Conditions: For resources that take time to provision, use wait conditions to ensure resources are fully available before proceeding.
-
Implement Error Handling: Use blocks with rescue and always sections to handle failures gracefully.
yaml- name: Create with error handling
block:
- name: Create instance
ec2_instance:
name: my-instance
# other parameters
rescue:
- name: Log failure
debug:
msg: "Failed to create instance"
always:
- name: Cleanup temp files
file:
path: /tmp/aws_temp
state: absent -
Version Control: Store your infrastructure code in version control systems like Git.
-
Use Delegation: Use
delegate_to: localhostfor cloud operations while still operating in a playbook that targets multiple hosts.
Troubleshooting Common Issues
Authentication Failures
If you encounter authentication errors:
- Verify your credentials are correct and not expired
- Check if you have the necessary permissions to perform the actions
- Ensure you're setting the correct region
Module Not Found Errors
If you get "module not found" errors:
fatal: [localhost]: FAILED! => {"msg": "The module amazon.aws.ec2_instance was not found"}
Make sure you've installed the required collection:
ansible-galaxy collection install amazon.aws
API Rate Limiting
If you hit rate limits:
- Add delay between tasks
- Use
throttleparameter for loops - Batch your operations when possible
Summary
Ansible Cloud Modules provide a powerful, consistent way to automate infrastructure across multiple cloud providers. By leveraging these modules, you can:
- Provision and manage cloud resources using the same Ansible syntax and patterns
- Create infrastructure as code that's version-controlled and repeatable
- Orchestrate complex multi-tier applications across cloud providers
- Automate the full lifecycle of your cloud resources
With the knowledge gained in this guide, you can begin automating your cloud infrastructure with Ansible, saving time and reducing manual configuration errors.
Additional Resources
- Ansible Collections Documentation
- Amazon AWS Collection Documentation
- Azure Collection Documentation
- Google Cloud Collection Documentation
Exercises
-
Basic AWS Deployment: Create a playbook to deploy an EC2 instance with a specific security group and tags.
-
Multi-Cloud Setup: Create a playbook that can deploy a similar VM on either AWS, Azure, or GCP based on a variable.
-
Cloud Inventory: Set up dynamic inventory for your cloud provider and create a playbook that operates only on instances with specific tags.
-
Auto-Scaling Group: Create a playbook that sets up an auto-scaling group in AWS with appropriate launch configurations and scaling policies.
-
Full Application Stack: Develop a complete playbook to deploy a three-tier application (web, application, database) on your preferred cloud provider.
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)