Make Me Happy: Easily Optimize AWS Organization with Terraform

Like the title says: it makes me happy to help others with things I’ve learned. I have a couple of techniques to Optimize AWS Organization with Terraform that bring efficiency and organization to managing my cloud organization.

In this guide, I’ll share those practical tips for effective AWS Organization management and Terraform use.

AWS Organization

A best practice for AWS is to organize your services into separate accounts for a variety of reasons:

• Group workloads based on business purpose and ownership
• Apply distinct security controls by environment.
• Constrain access to sensitive data.
• Promote innovation and agility.
• Limit scope of impact from adverse events.
• Support multiple IT operating models.
• Manage costs.

To make this easier AWS groups accounts with AWS Organizations, which is how all of the AWS accounts I work with are set up.

AWS Organization Terraform

In Terraform, you can manage these accounts using the aws provider, and the aws_organizations_organization data.

My account setup and organizational structure for this exercise is simple:

1. root – A root account – required and kept most secure where costs and the organization are managed.
2. manage – an account that has trust delegated to it for managing the organization.
3. central – a hub account used strictly for login and role assumption.
4. dev – account where development is done and things are tested.
5. prod – account for production code.

AWS Organization Hierarchy

I haven’t added any organizational hierarchy in the above. For an organization of any size I would use a more complex structure. I’d use organizational units (OU’s) to group accounts.

As an example, I would create an OU for each business unit. Each of those would have a series of production, development and staging OU’s. And each of those in turn would contain the various accounts for the needed services.

Getting Started

The first thing I need are credentials that Terraform can pick up from my environment (or secret manager like Vault). For that, I use standard AWS SSO to generate credentials for the manage account, and just paste them into my environment.

And I can verify that the above was successful by running a quick aws sts get-caller-identity to check the account ID is right.

AWS Organization Provider

The next thing I need is to connect those credentials to Terraform. To do that, I create provider for AWS. And I write the minimal provider shown below.

# Configure the AWS Provider
provider "aws" {
  alias  = "manage"
  region = "us-east-1"
}

In the code above alias lets me refer to that provider in other parts of the templates. This provider will be used to get the data to configure the one we do the work with.

Data Retrieval from AWS Organization

The next thing we’ll need is to get the accounts from the organization using the aws_organizations_organization data provider.

# Data source for the organization
data "aws_organizations_organization" "manage" {
    provider = aws.manage
}

Getting the account ID

To make this Terraform script run on a particular account (or even list of accounts), we need to pull the account ID from that organization data. To do that, I chose to use the account name as my Terraform workspace name. I make a local variable using the accounts from the data like this:

locals {
  # Get the current account ID using the workspace name to match the sub account name (only works if the accounts have the same name as the workspace)
  subaccount_id = [for each in data.aws_organizations_organization.manage.accounts : each.id if lower(each.name) == lower(terraform.workspace)][0]
}

This part of the template gets the account ID by looping through the accounts. Next, the list is filtered using the terraform.workspace environment variable. The result of that filtering is an array of 1 item. That element is assigned to the variable named subaccount_id

AWS Organization Sub-Account Provider

The subaccount_id is used Ito create a provider. This provider won’t have an alias, which makes it the default for anything else in the template. That means we don’t need to include provider in parts of the template acting on our subaccount.

# Assume into the account
provider "aws" {
  region     = "us-east-1"

  assume_role {
    # The role to assume in the sub account
    role_arn = "arn:aws:iam::${local.subaccount_id}:role/${var.role_name}"
  }
}

I’ve chosen to set the region to us-east-1. I used a variable for the role that the manage account has access to assume in the other accounts. The role_name variable would allow me to override which role I use at run time.

Terraform main.tf

With the provider set up, I can use the Terraform resource or data stanzas to get or update things in the specific account.

To complete this post, I decided to use a data provider to get some information about an IAM role as show below.

data "aws_iam_role" "AdminRole" {
  name               = "admin"
}

output "AdminRole" {
  description = "Admin Role Use Details"
    value = <<-EOT
    Role: ${data.aws_iam_role.AdminRole.id}
    Description: ${data.aws_iam_role.AdminRole.description}
    Id:   ${data.aws_iam_role.AdminRole.unique_id}
    Last Used Date: ${data.aws_iam_role.AdminRole.role_last_used[0].last_used_date}
    Last Used Region:${data.aws_iam_role.AdminRole.role_last_used[0].region}
    EOT
}

Terraform Setup

Now we have all the parts of the template for AWS Organization with Terraform. Next, I to create a workspace, I run the command terraform workspace new dev .

That switches me to the workspace dev. I can run a quick terraform plan to make sure I don’t have any typos.

I run terraform apply to do the final run. The only thing created is output variable

Output

To get the output, you can use a simple terraform output and a jq command as shown below

terraform output -json | jq -r ".AdminRole.value"
Role: admin
Description: 
Id:   AROAXXXXXXXXXX
Last Used Date: 2023-08-14T00:52:52Z
Last Used Region:us-east-1

Final Scripts

Explore the final templates for AWS Organization with Terraform, organized into variables.tf, providers.tf, and main.tf, showcasing best practices in structuring Terraform code for AWS Organization management.

variables.tf

Holds the variables for defaults. Variables can be overridden with the -var or -var-file command line argument.

variable "role_name" {
  type = string
    default = "MyAccessRole"
  description = "The role that will be assumed for this demo"
}

providers.tf

Holds the details for the providers we are going to use. I added some data stanzas to drive the AWS provider.

# Configure the AWS Provider
provider "aws" {
  alias  = "manage"
  region = "us-east-1"
}

# Data source for the organization
data "aws_organizations_organization" "manage" {
  provider = aws.manage
}

locals {
  # Get the current account ID using the workspace name to match the sub account name (only works if the accounts have the same name as the workspace)
  subaccount_id = [for each in data.aws_organizations_organization.manage.accounts : each.id if lower(each.name) == lower(terraform.workspace)][0]
}

# Assume into the account
provider "aws" {
  region     = "us-east-1"

  assume_role {
    # The role to assume in the sub account
    role_arn = "arn:aws:iam::${local.subaccount_id}:role/${var.role_name}"
  }
}

main.tf

By convention this is the main part of the template that holds what I am trying to get done with the providers.

data "aws_iam_role" "AdminRole" {
  name               = "admin"
}

output "AdminRole" {
  description = "Admin Role Use Details"
    value = <<-EOT
    Role: ${data.aws_iam_role.AdminRole.id}
    Description: ${data.aws_iam_role.AdminRole.description}
    Id:   ${data.aws_iam_role.AdminRole.unique_id}
    Last Used Date: ${data.aws_iam_role.AdminRole.role_last_used[0].last_used_date}
    Last Used Region:${data.aws_iam_role.AdminRole.role_last_used[0].region}
    EOT
}

Conclusion

This guide provides insights into leveraging AWS Organization with Terraform. By utilizing Terraform’s flexibility and AWS’s organizational capabilities, you can achieve a well-organized, scalable, and efficient cloud infrastructure..