Terraform Collection Functions
Introduction
Collection functions in Terraform allow you to manipulate and transform various types of collections - lists, maps, and sets. These functions are essential tools for infrastructure automation as they enable you to organize, filter, and transform your data efficiently while working with Terraform configurations.
In this guide, we'll explore the various collection functions available in Terraform, providing clear examples and practical applications to help you master these concepts.
What are Collections in Terraform?
Before diving into collection functions, let's understand what collections are in Terraform:
- Lists: Ordered sequences of values identified by position (index) starting from zero
- Maps: Collections of values where each is identified by a string label
- Sets: Unordered collections of unique values
Collection functions allow you to manipulate these data structures to suit your infrastructure requirements.
Common Collection Functions
1. concat - Combining Lists
The concat function combines two or more lists into a single list.
locals {
environments = ["dev", "staging"]
additional_environments = ["prod", "dr"]
}
output "all_environments" {
value = concat(local.environments, local.additional_environments)
# Result: ["dev", "staging", "prod", "dr"]
}
2. merge - Combining Maps
The merge function combines multiple maps into a single map. If there are duplicate keys, values from later maps will overwrite those from earlier ones.
locals {
default_tags = {
Environment = "development"
Owner = "DevOps Team"
}
custom_tags = {
Project = "Terraform Demo"
Environment = "testing" # This will override the Environment in default_tags
}
}
output "combined_tags" {
value = merge(local.default_tags, local.custom_tags)
/* Result:
{
Environment = "testing"
Owner = "DevOps Team"
Project = "Terraform Demo"
}
*/
}
3. flatten - Simplifying Nested Lists
The flatten function takes a list of lists and returns a flat list containing all the elements.
locals {
nested_list = [
["a", "b"],
["c", "d"],
["e", "f"]
]
}
output "flattened_list" {
value = flatten(local.nested_list)
# Result: ["a", "b", "c", "d", "e", "f"]
}
4. setproduct - Cartesian Product of Sets
The setproduct function computes the Cartesian product of multiple sets or lists.
locals {
environments = ["dev", "prod"]
regions = ["us-east-1", "eu-west-1"]
}
output "deployment_combinations" {
value = setproduct(local.environments, local.regions)
/* Result:
[
["dev", "us-east-1"],
["dev", "eu-west-1"],
["prod", "us-east-1"],
["prod", "eu-west-1"]
]
*/
}
5. contains - Checking for Element Presence
The contains function checks if a given list or set contains a specific value.
locals {
allowed_regions = ["us-east-1", "us-west-2", "eu-west-1"]
current_region = "us-west-2"
}
output "is_region_allowed" {
value = contains(local.allowed_regions, local.current_region)
# Result: true
}
6. keys and values - Extracting Map Components
The keys function returns a list of the keys in a map, while the values function returns a list of the values.
locals {
instance_types = {
dev = "t3.small"
test = "t3.medium"
prod = "m5.large"
}
}
output "environments" {
value = keys(local.instance_types)
# Result: ["dev", "test", "prod"]
}
output "instance_sizes" {
value = values(local.instance_types)
# Result: ["t3.small", "t3.medium", "m5.large"]
}
7. zipmap - Creating Maps from Lists
The zipmap function constructs a map from a list of keys and a list of values.
locals {
services = ["frontend", "backend", "database"]
ports = [80, 8080, 5432]
}
output "service_ports" {
value = zipmap(local.services, local.ports)
/* Result:
{
frontend = 80
backend = 8080
database = 5432
}
*/
}
Filtering and Transforming Collections
1. lookup - Safe Map Access with Default Value
The lookup function retrieves a value from a map for a specific key, with a default value if the key doesn't exist.
locals {
instance_types = {
dev = "t3.small"
test = "t3.medium"
prod = "m5.large"
}
}
output "staging_instance_type" {
value = lookup(local.instance_types, "staging", "t3.micro")
# Result: "t3.micro" (default value since "staging" key doesn't exist)
}
2. element - Accessing List Elements Safely
The element function retrieves an element from a list at a specific index, wrapping around if the index is out of bounds.
locals {
availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]
}
output "first_az" {
value = element(local.availability_zones, 0)
# Result: "us-east-1a"
}
output "wrapped_az" {
value = element(local.availability_zones, 4) # Index 4 wraps around to index 1
# Result: "us-east-1b"
}
3. chunklist - Splitting Lists into Fixed-Size Chunks
The chunklist function splits a list into fixed-size chunks, returning a list of lists.
locals {
subnets = ["subnet-1", "subnet-2", "subnet-3", "subnet-4", "subnet-5"]
}
output "subnet_pairs" {
value = chunklist(local.subnets, 2)
/* Result:
[
["subnet-1", "subnet-2"],
["subnet-3", "subnet-4"],
["subnet-5"]
]
*/
}
Advanced Collection Manipulations
Combining Functions for Complex Transformations
Collection functions can be combined to perform complex data transformations.
locals {
environments = ["dev", "staging", "prod"]
regions = ["us-east-1", "eu-west-1"]
# Generate all environment-region combinations
deployment_matrix = setproduct(local.environments, local.regions)
# Transform into map of objects for easier use
deployments = {
for pair in local.deployment_matrix :
"${pair[0]}-${pair[1]}" => {
environment = pair[0]
region = pair[1]
}
}
}
output "deployment_configurations" {
value = local.deployments
/* Result:
{
"dev-us-east-1" = {
environment = "dev"
region = "us-east-1"
},
"dev-eu-west-1" = {
environment = "dev"
region = "eu-west-1"
},
...and so on
}
*/
}
Practical Examples
Example 1: Dynamic Resource Creation with Collections
locals {
instance_config = {
web_server = {
instance_type = "t3.medium"
ami = "ami-0c55b159cbfafe1f0"
subnet_id = "subnet-abcdef123456"
},
app_server = {
instance_type = "t3.large"
ami = "ami-0c55b159cbfafe1f0"
subnet_id = "subnet-abcdef789012"
},
db_server = {
instance_type = "r5.large"
ami = "ami-0abcdef1234567890"
subnet_id = "subnet-abcdef345678"
}
}
}
resource "aws_instance" "servers" {
for_each = local.instance_config
ami = each.value.ami
instance_type = each.value.instance_type
subnet_id = each.value.subnet_id
tags = {
Name = each.key
}
}
Example 2: Filtering Resources Based on Environment
locals {
all_resources = {
vpc = {
cidr_block = "10.0.0.0/16"
env = ["dev", "staging", "prod"]
},
monitoring = {
enabled = true
env = ["staging", "prod"]
},
backup = {
retention_days = 30
env = ["prod"]
}
}
# Filter resources for the current environment
current_env = "staging"
env_resources = {
for name, config in local.all_resources :
name => config
if contains(config.env, local.current_env)
}
}
output "resources_for_environment" {
value = local.env_resources
/* Result for staging:
{
vpc = {
cidr_block = "10.0.0.0/16"
env = ["dev", "staging", "prod"]
},
monitoring = {
enabled = true
env = ["staging", "prod"]
}
}
*/
}
Example 3: Building Complex Network Configurations
locals {
vpc_cidr = "10.0.0.0/16"
subnets = {
public = {
azs = ["a", "b", "c"]
cidr = "10.0.0.0/20"
}
private = {
azs = ["a", "b", "c"]
cidr = "10.0.16.0/20"
}
database = {
azs = ["a", "b"]
cidr = "10.0.32.0/24"
}
}
# Generate all subnet configurations
subnet_configurations = flatten([
for subnet_type, config in local.subnets : [
for az_suffix in config.azs : {
name = "${subnet_type}-${az_suffix}"
az = "${data.aws_region.current.name}${az_suffix}"
cidr = cidrsubnet(config.cidr, ceil(log(length(config.azs), 2)), index(config.azs, az_suffix))
tier = subnet_type
}
]
])
}
output "subnet_config" {
value = local.subnet_configurations
/* Result will be a list of subnet configurations with CIDR blocks calculated
dynamically based on the size and number of subnets in each tier */
}
Collection Functions with for Expressions
Terraform's for expressions work alongside collection functions to provide powerful data manipulation capabilities:
Map Transformation
locals {
instances = {
web = "t3.micro"
app = "t3.small"
worker = "t3.medium"
}
# Transform map to add additional properties
enhanced_instances = {
for name, type in local.instances :
name => {
instance_type = type
is_production = contains(["app", "worker"], name)
name_upper = upper(name)
}
}
}
output "enhanced_instance_config" {
value = local.enhanced_instances
/* Result:
{
web = {
instance_type = "t3.micro"
is_production = false
name_upper = "WEB"
},
app = {
instance_type = "t3.small"
is_production = true
name_upper = "APP"
},
worker = {
instance_type = "t3.medium"
is_production = true
name_upper = "WORKER"
}
}
*/
}
List Transformation
locals {
users = ["alice", "bob", "charlie"]
user_objects = [
for user in local.users : {
username = user
email = "${user}@example.com"
admin = user == "alice" ? true : false
}
]
}
output "user_configuration" {
value = local.user_objects
/* Result:
[
{
username = "alice"
email = "[email protected]"
admin = true
},
{
username = "bob"
email = "[email protected]"
admin = false
},
{
username = "charlie"
email = "[email protected]"
admin = false
}
]
*/
}
Summary
Terraform collection functions are powerful tools that allow you to manipulate and transform lists, maps, and sets within your infrastructure code. These functions help you:
- Combine multiple collections using
concatandmerge - Extract specific elements using
elementandlookup - Transform collections with
flatten,zipmap, and others - Build complex data structures by combining multiple functions
When paired with Terraform's for expressions and conditionals, collection functions enable dynamic, flexible, and DRY infrastructure code that can adapt to various environments and requirements.
Exercises
To help solidify your understanding, try these exercises:
-
Create a configuration that generates a map of EC2 instances across three environments (dev, staging, prod) and two regions (us-east-1, us-west-2), with different instance types for each combination.
-
Build a networking configuration that dynamically creates the appropriate number of public and private subnets based on a list of availability zones.
-
Implement a tagging strategy that applies a base set of tags to all resources, but allows for resource-specific tag overrides.
Additional Resources
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)