Ansible Custom Modules
Introduction
Ansible is a powerful automation tool that uses modules to perform specific tasks on remote systems. While Ansible comes with hundreds of built-in modules, you might encounter situations where these don't quite meet your needs. This is where custom modules come in - they allow you to extend Ansible's functionality by writing your own modules tailored to your specific requirements.
In this guide, we'll explore how to create, test, and implement custom Ansible modules. By the end, you'll have the knowledge to extend Ansible's capabilities to fit your unique automation needs.
What Are Ansible Modules?
Before diving into custom modules, let's understand what Ansible modules are:
- Modules are reusable, standalone scripts that Ansible executes on target hosts
- They're the building blocks of Ansible playbooks
- Each module performs a specific task (e.g., managing files, users, packages, services)
- Modules accept parameters and return JSON data
When you run an Ansible playbook, it connects to your target hosts and pushes small programs called modules to them. These modules execute and return information back to the Ansible controller.
Why Create Custom Modules?
You might want to create custom modules when:
- You need to integrate with specific software not covered by existing modules
- You have complex logic that would be difficult to implement with existing modules
- You want to simplify playbooks by encapsulating repetitive tasks
- You need to interact with APIs or services that aren't supported out-of-the-box
Prerequisites
To follow along with this guide, you'll need:
- Basic understanding of Ansible concepts (playbooks, tasks, roles)
- Python programming knowledge (modules are written in Python)
- Ansible installed on your system
- A text editor or IDE
Creating Your First Custom Module
Let's create a simple custom module that checks if a specified process is running on the target host.
Step 1: Set Up the Development Environment
First, you'll need to determine where to place your custom modules. You have several options:
- In a specific project directory structure:
your_ansible_project/
├── library/ # Place custom modules here
├── playbooks/
└── inventory
- In a dedicated role:
your_ansible_project/
├── roles/
│ └── custom_modules/
│ └── library/ # Place custom modules here
- In Ansible's global module path (not recommended for development)
For this tutorial, we'll use the first approach.
Step 2: Create the Module File
Create a file named check_process.py
in your library/
directory:
#!/usr/bin/python
ANSIBLE_METADATA = {
'metadata_version': '1.1',
'status': ['preview'],
'supported_by': 'community'
}
DOCUMENTATION = '''
---
module: check_process
short_description: Check if a process is running
description:
- This module checks if a specified process is running on the target host.
options:
name:
description:
- Name of the process to check
required: true
type: str
author:
- Your Name (@yourgithub)
'''
EXAMPLES = '''
# Check if nginx is running
- name: Check nginx process
check_process:
name: nginx
# Check if apache2 is running and register the result
- name: Check apache2 process
check_process:
name: apache2
register: apache_status
'''
RETURN = '''
running:
description: Whether the process is running
type: bool
returned: always
pid:
description: Process ID if running, otherwise null
type: int
returned: always
'''
from ansible.module_utils.basic import AnsibleModule
import subprocess
def run_module():
# Define the module's parameters
module_args = {
'name': {'type': 'str', 'required': True}
}
# Create the module object
module = AnsibleModule(
argument_spec=module_args,
supports_check_mode=True
)
# Extract parameters
process_name = module.params['name']
# Initialize result dictionary
result = {
'changed': False,
'running': False,
'pid': None
}
# Check if we're in check mode
if module.check_mode:
module.exit_json(**result)
# Run the ps command to find the process
try:
ps_output = subprocess.check_output(
['ps', '-ef'],
universal_newlines=True
)
# Look for the process in the output
for line in ps_output.split('
'):
if process_name in line and 'grep' not in line:
# Process is running
result['running'] = True
# Extract PID (second column in ps output)
pid = int(line.split()[1])
result['pid'] = pid
break
# Return the result
module.exit_json(**result)
except subprocess.CalledProcessError as e:
module.fail_json(msg='Error checking process status', error=str(e), **result)
def main():
run_module()
if __name__ == '__main__':
main()
Step 3: Using Your Custom Module in a Playbook
Now, let's create a simple playbook to use our custom module:
---
- name: Test custom process check module
hosts: webservers
gather_facts: no
tasks:
- name: Check if nginx is running
check_process:
name: nginx
register: nginx_status
- name: Display result
debug:
msg: "Nginx is {{ 'running' if nginx_status.running else 'not running' }}{% if nginx_status.running %} with PID {{ nginx_status.pid }}{% endif %}"
Step 4: Testing the Module
You can test your module without running it against actual hosts using Ansible's ansible-test
command or manually with:
python -m ansible.modules.check_process name=nginx
For more thorough testing, you can also create a test environment with Docker or virtual machines.
Understanding Module Structure
Let's break down the key components of our custom module:
1. Module Metadata
ANSIBLE_METADATA = {
'metadata_version': '1.1',
'status': ['preview'],
'supported_by': 'community'
}
This provides information about the module's maturity and support level.
2. Documentation
DOCUMENTATION = '''
---
module: check_process
short_description: Check if a process is running
# ...rest of documentation...
'''
The documentation section is crucial - it helps users understand how to use your module. It should include:
- Module name and description
- All available parameters with their descriptions
- Required parameters
- Author information
3. Examples
EXAMPLES = '''
# Check if nginx is running
- name: Check nginx process
check_process:
name: nginx
# ...more examples...
'''
Providing clear examples helps users understand how to implement your module in their playbooks.
4. Return Values
RETURN = '''
running:
description: Whether the process is running
type: bool
returned: always
# ...more return values...
'''
This section documents what values your module will return, helping users understand what to expect and how to use the returned data.
5. The Module Logic
The core functionality lives in the run_module()
function:
- Define parameters the module accepts
- Create an AnsibleModule object
- Perform the actual work
- Return results
The key components to understand are:
- AnsibleModule: The main class that handles parameter validation, check mode, and more
- Result dictionary: Contains information about what the module did, including the
changed
flag - Exit functions:
module.exit_json()
for success andmodule.fail_json()
for failures
Advanced Custom Module Techniques
Now that we understand the basics, let's explore some more advanced techniques:
Complex Parameter Validation
For modules with many parameters or complex validation requirements, you can use more advanced parameter specifications:
module_args = {
'name': {'type': 'str', 'required': True},
'state': {
'type': 'str',
'required': False,
'default': 'present',
'choices': ['present', 'absent']
},
'timeout': {'type': 'int', 'required': False, 'default': 30},
'env_vars': {'type': 'dict', 'required': False}
}
Handling Check Mode
Check mode (--check flag) allows Ansible to simulate changes without actually making them. Proper handling ensures your module works correctly with this feature:
if module.check_mode:
# Predict what would happen but don't make changes
result['changed'] = would_change()
module.exit_json(**result)
# Normal mode code continues here...
Implementing Idempotency
A well-designed Ansible module should be idempotent - running it multiple times should result in the same system state:
# Check current state
current_state = get_current_state()
# Compare with desired state
if current_state != desired_state:
# Make changes
make_changes()
result['changed'] = True
else:
# No changes needed
result['changed'] = False
Using Module Utilities
Ansible provides helpful utilities to simplify common tasks:
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.common.dict_transformations import camel_dict_to_snake_dict
from ansible.module_utils.urls import fetch_url
These utilities can help with tasks like:
- API requests
- File operations
- Parameter handling
- JSON processing
Real-World Example: Custom API Module
Let's create a more practical module that interacts with a REST API. This module will create, update, or delete resources in a hypothetical service:
#!/usr/bin/python
ANSIBLE_METADATA = {
'metadata_version': '1.1',
'status': ['preview'],
'supported_by': 'community'
}
DOCUMENTATION = '''
---
module: custom_api
short_description: Manage resources in our API service
description:
- Create, update or delete resources in our custom API service
options:
endpoint:
description:
- API endpoint URL
required: true
type: str
api_key:
description:
- Authentication API key
required: true
type: str
resource_id:
description:
- ID of the resource to manage
required: false
type: str
state:
description:
- Desired state of the resource
choices: ['present', 'absent']
default: present
type: str
data:
description:
- Resource data (for creation or update)
required: false
type: dict
author:
- Your Name (@yourgithub)
'''
EXAMPLES = '''
# Create a new resource
- name: Create API resource
custom_api:
endpoint: "https://api.example.com/v1/resources"
api_key: "{{ api_key }}"
state: present
data:
name: "Test Resource"
type: "test"
properties:
color: blue
size: medium
register: result
# Update an existing resource
- name: Update API resource
custom_api:
endpoint: "https://api.example.com/v1/resources"
api_key: "{{ api_key }}"
resource_id: "12345"
state: present
data:
name: "Updated Resource"
properties:
color: red
register: result
# Delete a resource
- name: Delete API resource
custom_api:
endpoint: "https://api.example.com/v1/resources"
api_key: "{{ api_key }}"
resource_id: "12345"
state: absent
'''
RETURN = '''
resource:
description: The resource details
type: dict
returned: when state=present
status_code:
description: HTTP status code from the request
type: int
returned: always
changed:
description: Whether the resource was changed
type: bool
returned: always
'''
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.urls import fetch_url
import json
def run_module():
module_args = {
'endpoint': {'type': 'str', 'required': True},
'api_key': {'type': 'str', 'required': True, 'no_log': True},
'resource_id': {'type': 'str', 'required': False},
'state': {
'type': 'str',
'default': 'present',
'choices': ['present', 'absent']
},
'data': {'type': 'dict', 'required': False, 'default': {}}
}
result = {
'changed': False,
'status_code': None
}
module = AnsibleModule(
argument_spec=module_args,
supports_check_mode=True
)
endpoint = module.params['endpoint']
api_key = module.params['api_key']
resource_id = module.params['resource_id']
state = module.params['state']
data = module.params['data']
# Prepare headers
headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {api_key}'
}
# Check if resource exists for update/delete operations
if resource_id:
resource_url = f"{endpoint}/{resource_id}"
response, info = fetch_url(
module,
resource_url,
headers=headers,
method='GET'
)
result['status_code'] = info['status']
resource_exists = info['status'] == 200
# If running in check mode, predict the result
if module.check_mode:
if state == 'present' and not resource_exists:
result['changed'] = True
elif state == 'present' and resource_exists:
# Would update - assume it would change
result['changed'] = True
elif state == 'absent' and resource_exists:
result['changed'] = True
module.exit_json(**result)
# Actual execution
if state == 'absent' and resource_exists:
# Delete the resource
response, info = fetch_url(
module,
resource_url,
headers=headers,
method='DELETE'
)
result['status_code'] = info['status']
if info['status'] in [200, 202, 204]:
result['changed'] = True
else:
module.fail_json(
msg=f"Failed to delete resource: {info['msg']}",
**result
)
elif state == 'present' and resource_exists:
# Update existing resource
response, info = fetch_url(
module,
resource_url,
data=json.dumps(data),
headers=headers,
method='PUT'
)
result['status_code'] = info['status']
if info['status'] in [200, 201]:
content = response.read()
result['resource'] = json.loads(content.decode('utf-8'))
result['changed'] = True
else:
module.fail_json(
msg=f"Failed to update resource: {info['msg']}",
**result
)
# Create new resource
elif state == 'present':
# Skip actual creation in check mode
if module.check_mode:
result['changed'] = True
module.exit_json(**result)
# Create the resource
response, info = fetch_url(
module,
endpoint,
data=json.dumps(data),
headers=headers,
method='POST'
)
result['status_code'] = info['status']
if info['status'] in [200, 201]:
content = response.read()
result['resource'] = json.loads(content.decode('utf-8'))
result['changed'] = True
else:
module.fail_json(
msg=f"Failed to create resource: {info['msg']}",
**result
)
# Return results
module.exit_json(**result)
def main():
run_module()
if __name__ == '__main__':
main()
Using the API Module in a Playbook
---
- name: Manage API Resources
hosts: localhost
gather_facts: no
vars:
api_key: "your-secure-api-key"
tasks:
- name: Create a new resource
custom_api:
endpoint: "https://api.example.com/v1/resources"
api_key: "{{ api_key }}"
state: present
data:
name: "Production Server"
type: "server"
properties:
region: "us-west-2"
tier: "premium"
register: new_resource
- name: Show created resource
debug:
var: new_resource.resource
- name: Update the resource
custom_api:
endpoint: "https://api.example.com/v1/resources"
api_key: "{{ api_key }}"
resource_id: "{{ new_resource.resource.id }}"
state: present
data:
properties:
tier: "standard"
register: updated_resource
Best Practices for Custom Modules
To ensure your modules are robust, maintainable, and user-friendly:
1. Follow Ansible Module Conventions
- Use consistent parameter naming
- Follow Python PEP 8 style guidelines
- Include comprehensive documentation
- Provide meaningful examples
2. Handle Errors Gracefully
try:
# Attempt some operation
result = do_something_risky()
except Exception as e:
module.fail_json(
msg="Operation failed",
error=str(e),
**result
)
3. Ensure Idempotency
- Check current state before making changes
- Only report
changed=True
when actual changes occur - Make your module repeatable without side effects
4. Support Check Mode
Always implement check mode support to allow users to test playbooks safely:
if module.check_mode:
result['changed'] = would_change()
module.exit_json(**result)
5. Secure Sensitive Data
- Use
no_log: True
for sensitive parameters:
module_args = {
'password': {'type': 'str', 'required': True, 'no_log': True}
}
6. Return Useful Information
- Return data that helps users understand what happened
- Include status information, IDs, and other relevant details
- Maintain consistency in return values
Distributing Custom Modules
Once you've created useful custom modules, you might want to share them with your team or the community:
1. Within Your Organization
- Create a dedicated Ansible role containing your modules
- Use git repositories to version control your modules
- Document usage with examples
2. As an Ansible Collection
For more formal distribution, consider creating an Ansible Collection:
collection/
├── docs/
├── galaxy.yml
├── plugins/
│ └── modules/
│ ├── check_process.py
│ └── custom_api.py
├── README.md
└── roles/
Build and distribute with:
ansible-galaxy collection build
ansible-galaxy collection publish ./my_namespace-my_collection-1.0.0.tar.gz
Debugging Custom Modules
When developing modules, debugging is crucial. Here are some techniques:
1. Enable Ansible Debug Mode
ANSIBLE_DEBUG=1 ansible-playbook -v your_playbook.yml
2. Add Debug Output
import json
# Debug information
debug_info = {
'params': module.params,
'current_state': current_state
}
# Write to a file
with open('/tmp/ansible_debug.json', 'w') as f:
f.write(json.dumps(debug_info, indent=4))
3. Using Python Debugger
For local testing, you can add breakpoints:
import pdb
def run_module():
# ... code ...
pdb.set_trace() # This will start the debugger
# ... more code ...
Summary
Creating custom Ansible modules allows you to extend Ansible's capabilities to suit your specific needs. By following best practices and understanding the module structure, you can create robust, reusable automation components.
In this guide, we've covered:
- The basics of Ansible modules
- How to create a simple custom module
- Advanced techniques for parameter handling and API interactions
- Best practices for module development
- Debugging and distribution methods
Next Steps
Now that you understand how to create custom Ansible modules, consider these next steps:
- Practice: Convert some of your complex playbook tasks into custom modules
- Explore: Study the source code of existing Ansible modules for inspiration
- Contribute: Share your modules with the Ansible community
- Integrate: Combine your modules with roles and collections for complete solutions
Exercises
- Create a custom module that checks if a specific file contains a pattern and optionally replaces it
- Develop a module that interacts with a database (MySQL, PostgreSQL, etc.)
- Build a module that integrates with a cloud service not covered by existing modules
- Enhance the API module from this guide to handle pagination and error retries
Additional Resources
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)