Terraform for_each with Index

Terraform is currently one of the most popular tools for implementing infrastructure as code (IaC) for cloud and on-premises infrastructures. Its feature for_each loop, allows users to describe and manipulate many resources simultaneously. In contrast, in a configuration. Although at some times, it is critical to index each resource, this is where for_each along with the use of index values comes in handy. In this guide, we deep dive into understanding for_each, how it differs from count, using for_each with maps for indexed resources, and the do's and don’ts for using for_each with index.

The for_each is a meta-argument in Terraform that allows users to generate and operate multiple instances of a resource or module depending on input values. While for_each and count are meta-arguments in the Terraform, they are different from regular loops in many programming languages For_each is an effective opportunity to loop over datasets like lists and maps. In contrast, count helps to control the amount of created resources at once.For_each is useful to manage infrastructure resources separately, and the unique identifiers of every created resource are its main advantage.

Indexing is when in a loop, an index is attached to each resource to identify it. Count can dispense with integer subscripts such as 0, 1, and 2 but for_each with custom indexes is better when more control is needed. This sort of control is very effective if you desire to point a unique ID, key or value to a certain base in a given set.

Syntax of for_each

In Terraform, the for_each meta-argument allows you to create multiple instances of a resource or module by iterating over a collection (like a map, set, or list). Here’s the basic format:

resource "<RESOURCE_TYPE>" "<NAME>" {
  for_each = <COLLECTION>
  
  # Access collection keys or values
  key_attribute   = each.key
  value_attribute = each.value
  
  # Other resource attributes
}

• for_each: Specifies the collection (such as a map or set) to iterate over.
• each.key: Refers to the key from the collection being processed.
• each.value: Refers to the value corresponding to the key.

Terraform for_each Object

The for_each object in Terraform refers to the current item being processed during an iteration. This object has two key attributes:

• each.key: Represents the key of the current item in the collection.
• each.value: Represents the corresponding value

Structure of the for_each Object

{
  key   = <KEY>
  value = <VALUE>
}

Example:

variable "subnets" {
  default = {
    "subnet1" = "10.0.1.0/24"
    "subnet2" = "10.0.2.0/24"
  }
}

resource "aws_subnet" "example" {
  for_each   = var.subnets
  cidr_block = each.value  # Uses the value from the map
  tags = {
    Name = each.key         # Uses the key as the tag
  }
}

Here:

• each.key represents the subnet name (subnet1, subnet2).
• each.value provides the corresponding CIDR block (10.0.1.0/24, 10.0.2.0/24).

Terraform for_each vs count

For_each and count can make more than one instance of a resource but they have different use and workings. Knowing the differences between them is the foundation to apply for_each with indexing in Terraform.

1. When to use count

• Uniform Resources: When all instances of a resource are identical.
• Automatic Indexing: Automatically assigns resources an index (0, 1, 2, etc.).
• Limited Customization: Limited ability to assign unique properties to each instance.

For example, count is ideal when creating identical instances of a VM with no significant customizations between them.

resource "aws_instance" "example" {
  count = 3
  ami           = "ami-123456"
  instance_type = "t2.micro"
}

2. When to use for_each

• Unique Identifiers: Enables custom resource keys, making it ideal for named or identified resources.
• Dynamic Properties: Each resource can have unique attributes.
• Better with Maps and Sets: Works effectively with maps and sets, which are more flexible than lists.

With for_each, you can assign a unique identifier for each resource, allowing easier tracking and management.

resource "aws_instance" "example" {
  for_each = { "web1" = "ami-123456", "web2" = "ami-789012" }
  ami           = each.value
  instance_type = "t2.micro"
}

Using Terraform for_each with Map for Indexed Resources

We mentioned that maps, a data source in Terraform, are friendly when used with for_each because you can define a unique key for each resource. In this regards, indexing means that each resource should be unique and recognizable by map key to ensure effective implementation and storage.

Step-by-Step Guide for Using for_each with Map for Indexing

• Define the Map: The next order of business was to define a map for storing the resource identifier and its values. Each key refers to a distinct tag, so every resource holds an index that may be used to access it.

variable "instance_details" {
  type    = map(string)
  default = {
    "web1" = "ami-123456"
    "web2" = "ami-789012"
  }
}

• Use the Map in for_each: In the resource block, set for_each to the map. Terraform will therefore iterate through each of the key value pair in a map and create a new resource for each one of them.

resource "aws_instance" "example" {
  for_each      = var.instance_details
  ami           = each.value
  instance_type = "t2.micro"
  tags = {
    Name = each.key
  }
}

• Referencing Resources with Indexes: You can reference each resource by its unique key, each.key, within the loop. The syntax aws_instance.example["web1"].id will retrieve the specific instance with the identifier web1.

Practical Examples of for_each with Index in Terraform

Below are some practical examples that demonstrate how for_each can be combined with index-like identifiers for flexible resource creation.

Example 1: Creating Multiple S3 Buckets with Unique Tags

Imagine you need to create S3 buckets with unique tags for each department in an organization. You can use for_each with a map where each key represents a department.

variable "departments" {
  type = map(string)
  default = {
    "finance"   = "finance-bucket"
    "marketing" = "marketing-bucket"
    "hr"        = "hr-bucket"
  }
}

resource "aws_s3_bucket" "department_buckets" {
  for_each = var.departments
  bucket   = each.value
  tags = {
    Department = each.key
  }
}

Example 2: Defining a List of Subnets with Custom CIDR Blocks

For example, if you were configuring a group of Internet subnets for your system that is distributed across multiple availability zones you have to assign a different CIDR block to every one of them. In this case, you can use for_each to have each key that relates to a particular AZ in the map and each value corresponds to a CIDR block.

variable "subnets" {
  type = map(string)
  default = {
    "us-east-1a" = "10.0.1.0/24"
    "us-east-1b" = "10.0.2.0/24"
  }
}

resource "aws_subnet" "example_subnet" {
  for_each = var.subnets
  vpc_id   = "vpc-123456"
  cidr_block = each.value
  availability_zone = each.key
}

Best Practices and Common Pitfalls When Using for_each with Index

To use for_each effectively in Terraform, it’s important to understand its structure and follow best practices while being aware of common pitfalls. Here’s what you should keep in mind:

1. Best Practices

• Use Maps and Sets for Unique Identifiers: When creating multiple resources, prefer using maps or sets over lists. These data types allow for flexible indexing, which is crucial when managing multiple resources with distinct properties.
• Keep Keys Consistent: Ensure that the keys you use in maps are unique and meaningful. The key should clearly identify the resource it corresponds to, making it easier to manage and reference later.
• Reference Resources Clearly: Always use the key to refer to your resources in a way that helps you track them easily. By doing this, you can maintain better clarity, especially when scaling your infrastructure.

2. Common Pitfalls

• Avoid Mixing count and for_each: Using both count and for_each in the same resource block can cause confusion and lead to unexpected results. Stick to one or the other to avoid complicating your Terraform configuration.
• Be Careful When Changing Keys in Maps: Modifying keys or values in maps after resources have been created can lead to Terraform recreating those resources. This can be disruptive, especially in production environments, so always try to plan changes carefully.
• Handle Dependencies Properly: When using for_each with resources that have dependencies, make sure to define those dependencies correctly. If not, Terraform may create resources in the wrong order, causing issues in your infrastructure setup.

Key Benefits of Using for_each with Index

• Enables explicit resource identification, especially when creating resources with specific keys.
• Facilitates dynamic creation and referencing of resources.
• Provides more flexibility than count, as you can use maps with custom keys.

Limitations of Using for_each with Index

Although for_each with an index can be incredibly useful in Terraform, there are some limitations and challenges to consider:

• No Automatic Indexing for Non-Map Collections: When using for_each with lists, Terraform doesn't automatically assign a custom index for each element. Unlike the count argument, which creates an index (0, 1, 2, etc.) for you, for_each requires a map or set for custom indexing. This means if you need more control over how your resources are identified, using for_each with a list can be a bit tricky.
• Reordering Resources Can Cause Issues: If you reorder items in your map, Terraform will treat it as a change in resource identifiers and destroy and recreate those resources. This can lead to unnecessary downtime, particularly in production environments, because the keys are tightly linked to specific resources.
• Limited Flexibility with Complex Data Structures: for_each is great with maps and sets, but it’s not as effective when you need to iterate over more complex structures. If your resources have nested attributes or require combining multiple variables, for_each can become cumbersome and harder to manage.
• Increased Complexity for Large-Scale Projects: The more resources you create with for_each, the more complicated your configuration can get. As your infrastructure grows, managing the relationships between resources, especially when they are indexed, can become harder to track and maintain.
• Challenges with Dynamic Resources: If your resources are created based on dynamic values (such as outputs from other resources), using for_each with custom indexing might cause issues. Terraform can have trouble managing dynamic changes in the state, leading to problems with dependencies or unexpected updates.
• Reduced Readability in Complex Configurations: While using for_each with indices allows flexibility, it can also make your code harder to understand, especially for those unfamiliar with it. It may be more difficult to determine which key corresponds to which resource, leading to a less readable and harder-to-debug configuration.

Conclusion

So when we use for_each with indexing through maps in Terraform we get the flexibility which make it easier to manage dynamic resources. The for_each is useful in complex environments where there are a variety of resources which may have specific requirements because for_each uses distinct keys to define the resources attributes and this is easy to refer to. This topic helps you make the right distinction between for_each and count, maximize the use of maps, and adhere to the best practices in writing reusable and adaptive Terraform code to maintain the state of your infrastructure.