Skip to main content

Ansible GCP Integration

Introduction

Ansible is a powerful open-source automation tool that allows you to configure systems, deploy software, and orchestrate advanced IT tasks. Google Cloud Platform (GCP) is one of the leading cloud providers that offers a wide range of services including compute, storage, networking, and more.

The integration between Ansible and GCP enables you to automate the provisioning, configuration, and management of your GCP resources using Ansible's declarative approach. This means you can define the desired state of your GCP infrastructure in simple YAML files and let Ansible handle the implementation details.

In this guide, we'll explore how to use Ansible for GCP automation, set up the necessary requirements, and work through practical examples that demonstrate the power of this integration.

Prerequisites

Before we begin, you'll need:

  • Ansible installed (version 2.9 or higher)
  • Python 3.6 or higher
  • GCP account with a project set up
  • Basic knowledge of Ansible concepts (playbooks, modules, inventories)
  • Basic understanding of GCP services

Setting Up Ansible for GCP

1. Install Required Packages

First, let's install the necessary Python packages:

bash
pip install ansible google-auth requests

2. Install GCP Collection

Ansible provides a collection specifically for GCP:

bash
ansible-galaxy collection install google.cloud

3. Set Up Authentication

Ansible needs to authenticate with GCP. The recommended approach is using a service account:

  1. Create a service account in the GCP Console
  2. Assign appropriate roles to the service account
  3. Download the service account key file (JSON)
  4. Set the environment variable or configure the auth in your playbook:
bash
export GCP_SERVICE_ACCOUNT_FILE=/path/to/your-service-account-file.json

Basic GCP Module Usage

Let's start with a simple example of managing a GCP Compute Engine instance:

yaml
---
- name: Create a GCP VM instance
hosts: localhost
gather_facts: false
vars:
gcp_project: your-project-id
gcp_cred_kind: serviceaccount
gcp_cred_file: /path/to/your-service-account-file.json
zone: us-central1-a
machine_type: e2-medium

tasks:
- name: Create a disk
google.cloud.gcp_compute_disk:
name: "ansible-disk"
size_gb: 10
source_image: "projects/debian-cloud/global/images/family/debian-10"
zone: "{{ zone }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: disk

- name: Create a VM instance
google.cloud.gcp_compute_instance:
name: "ansible-instance"
machine_type: "{{ machine_type }}"
disks:
- auto_delete: true
boot: true
source: "{{ disk }}"
network_interfaces:
- network: null # Uses the default network
access_configs:
- name: "External NAT"
nat_ip: null
type: "ONE_TO_ONE_NAT"
zone: "{{ zone }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: instance

- name: Show instance details
debug:
var: instance

This playbook:

  1. Creates a disk with Debian 10 OS image
  2. Provisions a VM instance using the created disk
  3. Displays the resulting instance details

Using Dynamic Inventory with GCP

Ansible can dynamically discover and inventory your GCP resources, making it easier to manage large-scale deployments.

Setting Up Dynamic Inventory

Create a file named gcp.yml with the following content:

yaml
plugin: google.cloud.gcp_compute
projects:
- your-project-id
zones:
- us-central1-a
- us-central1-b
hostnames:
- name
filters: []
auth_kind: serviceaccount
service_account_file: /path/to/your-service-account-file.json

You can then use this inventory with Ansible commands:

bash
ansible-inventory -i gcp.yml --list

And in playbooks:

yaml
---
- name: Manage GCP instances
hosts: all
gather_facts: false
tasks:
- name: Install Apache
apt:
name: apache2
state: present
become: true
when: "'apache-servers' in group_names"

Managing GCP Resources

Let's look at a more comprehensive example that manages multiple GCP resources:

Creating a Complete GCP Environment

yaml
---
- name: Set up a complete GCP environment
hosts: localhost
gather_facts: false
vars:
gcp_project: your-project-id
gcp_cred_kind: serviceaccount
gcp_cred_file: /path/to/your-service-account-file.json
region: us-central1
zone: us-central1-a

tasks:
# Create a VPC Network
- name: Create a custom VPC network
google.cloud.gcp_compute_network:
name: "ansible-network"
auto_create_subnetworks: false
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: network

# Create a Subnet
- name: Create a subnet
google.cloud.gcp_compute_subnetwork:
name: "ansible-subnet"
region: "{{ region }}"
network: "{{ network }}"
ip_cidr_range: "10.0.0.0/24"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: subnet

# Create a Firewall Rule
- name: Allow HTTP traffic
google.cloud.gcp_compute_firewall:
name: "allow-http"
network: "{{ network }}"
allowed:
- ip_protocol: tcp
ports:
- '80'
source_ranges:
- '0.0.0.0/0'
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present

# Create Multiple VM Instances
- name: Create VMs for web application
google.cloud.gcp_compute_instance:
name: "web-server-{{ item }}"
machine_type: "e2-medium"
disks:
- auto_delete: true
boot: true
initialize_params:
source_image: "projects/debian-cloud/global/images/family/debian-10"
disk_size_gb: 10
network_interfaces:
- network: "{{ network }}"
subnetwork: "{{ subnet }}"
access_configs:
- name: "External NAT"
nat_ip: null
type: "ONE_TO_ONE_NAT"
tags:
items:
- http-server
- web-server
metadata:
items:
- key: startup-script
value: |
#!/bin/bash
apt-get update
apt-get install -y apache2
echo "Hello from Ansible-managed GCP VM $(hostname)" > /var/www/html/index.html
zone: "{{ zone }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: instances
loop:
- 1
- 2
- 3

# Create a Load Balancer
- name: Create a health check
google.cloud.gcp_compute_health_check:
name: "http-health-check"
type: HTTP
http_health_check:
port: 80
request_path: "/"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: health_check

- name: Create a backend service
google.cloud.gcp_compute_backend_service:
name: "web-backend-service"
health_checks:
- "{{ health_check }}"
backends:
- group: "{{ item.selfLink }}"
balancing_mode: "UTILIZATION"
capacity_scaler: 1.0
protocol: "HTTP"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: backend_service
with_items: "{{ instances.results }}"

This playbook creates a complete environment including:

  • Custom VPC network
  • Subnet
  • Firewall rules
  • Multiple VM instances with Apache installed
  • Load balancer configuration

Using Ansible Variables and Templates with GCP

Ansible's templating capabilities allow us to create more dynamic and reusable playbooks:

Creating a GCP Infrastructure Template

First, create a template file named instance_template.j2:

jinja
---
name: "{{ instance_name }}"
machine_type: "{{ machine_type }}"
disks:
- auto_delete: true
boot: true
initialize_params:
source_image: "{{ image }}"
disk_size_gb: {{ disk_size }}
network_interfaces:
- network: null
access_configs:
- name: "External NAT"
nat_ip: null
type: "ONE_TO_ONE_NAT"
{% if tags %}

items:
{% for tag in tags %}
- {{ tag }}
{% endfor %}
{% endif %}
zone: "{{ zone }}"

Then, use this template in your playbook:

yaml
---
- name: Deploy from template
hosts: localhost
gather_facts: false
vars:
gcp_project: your-project-id
gcp_cred_kind: serviceaccount
gcp_cred_file: /path/to/your-service-account-file.json
zone: us-central1-a
instances:
- name: web-1
machine_type: e2-medium
disk_size: 10
tags:
- web
- production
- name: web-2
machine_type: e2-medium
disk_size: 10
tags:
- web
- production
- name: db-1
machine_type: e2-standard-2
disk_size: 50
tags:
- database
- production

tasks:
- name: Create instances from template
google.cloud.gcp_compute_instance:
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
name: "{{ item.name }}"
machine_type: "{{ item.machine_type }}"
disks:
- auto_delete: true
boot: true
initialize_params:
source_image: "projects/debian-cloud/global/images/family/debian-10"
disk_size_gb: "{{ item.disk_size }}"
network_interfaces:
- network: null
access_configs:
- name: "External NAT"
nat_ip: null
type: "ONE_TO_ONE_NAT"
tags:
items: "{{ item.tags }}"
zone: "{{ zone }}"
loop: "{{ instances }}"

Working with Cloud Storage

Ansible can also manage GCP Storage resources:

yaml
---
- name: Manage GCP Storage
hosts: localhost
gather_facts: false
vars:
gcp_project: your-project-id
gcp_cred_kind: serviceaccount
gcp_cred_file: /path/to/your-service-account-file.json

tasks:
# Create a bucket
- name: Create a GCS bucket
google.cloud.gcp_storage_bucket:
name: "ansible-gcp-bucket-{{ gcp_project }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: bucket

# Upload a file
- name: Create a local file
copy:
content: "This is a test file created by Ansible"
dest: "/tmp/test-file.txt"

- name: Upload file to bucket
google.cloud.gcp_storage_object:
bucket: "{{ bucket.name }}"
src: "/tmp/test-file.txt"
dest: "uploaded-by-ansible/test-file.txt"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present

Visualizing Ansible GCP Integration

Let's create a simple diagram to illustrate how Ansible integrates with GCP:

Best Practices for Ansible GCP Integration

When working with Ansible and GCP, consider the following best practices:

  1. Use Service Accounts with Limited Permissions

    • Follow the principle of least privilege
    • Create different service accounts for different environments
  2. Organize Your Playbooks

    • Keep tasks modular and reusable
    • Use roles for complex infrastructure setups
  3. Use Variables and Templates

    • Store environment-specific variables separately
    • Template complex configurations
  4. Manage State

    • Be careful with state: absent operations
    • Use --check mode to preview changes
  5. Version Control Everything

    • Store playbooks in a version control system
    • Use tags to mark stable versions
  6. Use Dynamic Inventory

    • Leverage dynamic inventory for large-scale deployments
    • Group resources logically

Example: Complete Web Application Deployment

Let's put everything together to deploy a complete web application:

yaml
---
- name: Deploy Web Application to GCP
hosts: localhost
gather_facts: false
vars:
gcp_project: your-project-id
gcp_cred_kind: serviceaccount
gcp_cred_file: /path/to/your-service-account-file.json
region: us-central1
zone: us-central1-a
app_name: my-web-app
instance_count: 2

tasks:
# Create VPC
- name: Create VPC network
google.cloud.gcp_compute_network:
name: "{{ app_name }}-network"
auto_create_subnetworks: false
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: network

# Create subnet
- name: Create subnet
google.cloud.gcp_compute_subnetwork:
name: "{{ app_name }}-subnet"
region: "{{ region }}"
network: "{{ network }}"
ip_cidr_range: "10.0.0.0/24"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: subnet

# Create firewall rules
- name: Create firewall rules
google.cloud.gcp_compute_firewall:
name: "{{ app_name }}-{{ item.name }}"
network: "{{ network }}"
allowed: "{{ item.allowed }}"
source_ranges: "{{ item.source_ranges }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
loop:
- name: allow-http
allowed:
- ip_protocol: tcp
ports:
- '80'
source_ranges:
- '0.0.0.0/0'
- name: allow-ssh
allowed:
- ip_protocol: tcp
ports:
- '22'
source_ranges:
- '0.0.0.0/0'

# Create instance template
- name: Create instance template
google.cloud.gcp_compute_instance_template:
name: "{{ app_name }}-template"
properties:
machine_type: e2-medium
disks:
- auto_delete: true
boot: true
initialize_params:
source_image: "projects/debian-cloud/global/images/family/debian-10"
network_interfaces:
- network: "{{ network }}"
subnetwork: "{{ subnet }}"
access_configs:
- name: "External NAT"
type: "ONE_TO_ONE_NAT"
metadata:
items:
- key: startup-script
value: |
#!/bin/bash
apt-get update
apt-get install -y apache2 git
git clone https://github.com/example/web-app.git /var/www/html/
systemctl restart apache2
tags:
items:
- http-server
- web-app
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: template

# Create instance group
- name: Create managed instance group
google.cloud.gcp_compute_instance_group_manager:
name: "{{ app_name }}-igm"
base_instance_name: "{{ app_name }}"
instance_template: "{{ template }}"
target_size: "{{ instance_count }}"
zone: "{{ zone }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: instance_group

# Create health check
- name: Create HTTP health check
google.cloud.gcp_compute_health_check:
name: "{{ app_name }}-health-check"
type: HTTP
http_health_check:
port: 80
request_path: "/"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: health_check

# Create backend service
- name: Create backend service
google.cloud.gcp_compute_backend_service:
name: "{{ app_name }}-backend"
backends:
- group: "{{ instance_group.instanceGroup }}"
health_checks:
- "{{ health_check }}"
protocol: "HTTP"
region: "{{ region }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: backend_service

# Create URL map
- name: Create URL map
google.cloud.gcp_compute_url_map:
name: "{{ app_name }}-url-map"
default_service: "{{ backend_service }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: url_map

# Create HTTP proxy
- name: Create HTTP proxy
google.cloud.gcp_compute_target_http_proxy:
name: "{{ app_name }}-http-proxy"
url_map: "{{ url_map }}"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: http_proxy

# Create forwarding rule
- name: Create forwarding rule
google.cloud.gcp_compute_global_forwarding_rule:
name: "{{ app_name }}-forwarding-rule"
target: "{{ http_proxy }}"
port_range: "80-80"
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: forwarding_rule

# Output the public IP
- name: Display app URL
debug:
msg: "Application deployed! Access it at: http://{{ forwarding_rule.IPAddress }}"

This playbook demonstrates a complete web application deployment including:

  1. Network infrastructure
  2. Instance templates and managed instance groups
  3. Load balancing
  4. Health checks
  5. HTTP proxies and forwarding rules

Troubleshooting Ansible GCP Integration

If you encounter issues with your Ansible GCP integration, consider these troubleshooting steps:

  1. Authentication Issues

    • Verify service account credentials are correct
    • Check permissions of the service account
    • Validate environment variables
  2. Module Errors

    • Ensure you're using the correct module version
    • Check parameter syntax and requirements
    • Review GCP API quotas and limits
  3. Debugging Tips

    • Use the -vvv flag for verbose output
    • Check GCP Activity Logs for API errors
    • Use register and debug tasks to inspect variables

Example debugging playbook:

yaml
---
- name: Debug GCP Connection
hosts: localhost
gather_facts: false
vars:
gcp_project: your-project-id
gcp_cred_kind: serviceaccount
gcp_cred_file: /path/to/your-service-account-file.json

tasks:
- name: Verify authentication
google.cloud.gcp_compute_disk_info:
zone: us-central1-a
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
register: result

- name: Display result
debug:
var: result
verbosity: 0

Summary

In this guide, we've explored how to integrate Ansible with Google Cloud Platform to automate cloud infrastructure management. We've covered:

  • Setting up Ansible for GCP
  • Creating basic GCP resources with Ansible modules
  • Using dynamic inventory for managing GCP instances
  • Deploying complete infrastructure stacks
  • Using templates and variables for more flexible deployments
  • Managing cloud storage resources
  • Best practices and troubleshooting tips

Ansible's integration with GCP provides a powerful way to manage cloud resources using Infrastructure as Code principles, enabling you to create reproducible, version-controlled, and automated cloud deployments.

Additional Resources

To further your learning about Ansible GCP integration, consider these resources:

  1. Ansible GCP Collection Documentation
  2. Google Cloud Platform Documentation
  3. Ansible Best Practices

Exercises

To practice your skills, try these exercises:

  1. Create a playbook that deploys a three-tier web application (web, application, database) on GCP.
  2. Modify the example playbooks to use Ansible Vault for securing GCP credentials.
  3. Create a dynamic inventory script that groups GCP instances by their tags or labels.
  4. Implement a blue-green deployment strategy using Ansible and GCP instance groups.
  5. Automate the backup of a GCP VM instance using snapshots and scheduled Ansible playbooks.


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)