Managing AWS Resource Tags

AWS resource tags are a great way to manage, identify, organize, search, and filter your cloud resources, especially when you have thousands of them across different business units and teams. A resource tag is simply a key-value pair in the resource metadata that can be set while creating the resource or any time later.

What are their use cases

Tags are pretty useful for resource organization, cost allocation, automation and access control use cases.

For resource organization purposes; tags can be utilized to categorize resources by purpose, owner, environment, or any other need. So that it is possible to filter resources in AWS console or while using API operations to list or alter a subset of resources.

For cost allocation purposes; tags allow to break down costs in AWS Cost Explorer. There can be multiple strategies to leverage tags to allocate cloud costs. The first one is using business related tags such as business unit, team and project tags to attribute costs to particular teams and projects. Another one is to use technical tags such as environment tags to distinguish costs of different environments or name tags to view the costs occurred by a particular resource.

For automation purposes; tags help us to identify resources to run our automation cases. For example tagging development instances such aspurpose=dev enables to run an automation every night to shut down all development resources which are left open to avoid incurring costs.

For access control purposes; tags can be used to allow or deny access for a group of resources as the IAM policies support tag-based conditions. This integration is powerful and enables users and roles to access environment, project and/or team specific resources dynamically.

How to keep them under control

The governance of the tags are not an easy task. It is pretty easy to end up with untagged resources and different tags across teams with the same meaning or even typos.

The easiest way to ensure the resources have their tags is by ensuring they are tagged while being created. The IaC solutions like AWS CloudFormation and Terraform help to remember and systematically apply them, also enables reviewers to check them. It is also possible to enforce having several tags using SCP policies, check this example to require a tag while a resource is being created. This will not permit the creation of a resource that is not compliant with the policy.

There are times when a bulk update of the resource tags is required. There may be many reasons such as tagging previously untagged resources, renaming a tag, updating a tag value, introducing a brand new tag to all resources etc.

AWS provides Tag Editor console and Resource Groups Tagging API for the bulk operations. Every service has their own API for tag modification as well, but there is no standard interface among services, therefore, it can be painful to use them separately.

While these two services are pretty useful to search and tag a subset of resources, there are multiple operations required to do so. Also it is not possible to set different tag values while updating multiple resources. This is where the aws-tag command line tool comes in handy. You can install it using pip. Note that this is not developed, related or owned by AWS, but an open source tool.

pip install aws-tag

The aws-tag utility helps to list and tag a subset of resources using tag filters in a simple way. Moreover, it enables exporting and importing csv files to bulk tag multiple resources across different AWS services using different tag keys and values, which is the key differentiation to available AWS tools.

Listing resources based on tags can be achieved easily in both tools. See the example below to find DynamoDB tables that have team=data and environment=production tags.

# AWS Resource Groups Tagging API
aws resourcegroupstaggingapi get-resources \
    --resource-type-filters dynamodb:table \
    --tag-filters 'Key=team','Values=data' \
    --tag-filters 'Key=environment','Values=production'

# aws-tag
aws-tag list \
    --service dynamodb \
    --filter 'team=data' \
    --filter 'environment=production'

The more powerful operations show up in a scenario when we want to filter and tag same group of resources. See the example below to add subteam=intelligence tag for Kinesis Data Streams resources that have team=data tag and the resource name starting with intel-.

# AWS Resource Groups Tagging API
arns=$(aws resourcegroupstaggingapi get-resources \
    --resource-type-filters 'kinesis:stream'\
    --tag-filters 'Key=team','Values=data' \
    --query 'ResourceTagMappingList[*].ResourceARN' \
    | jq -r '.[]')

for arn in $arns; do
    resource_name=$(echo $arn | awk -F '/' '{print $2}')

    if [[ $resource_name == intel-* ]]; then
        aws resourcegroupstaggingapi tag-resources \
            --resource-arn-list $arn \
            --tags 'subteam=intelligence'
    fi
done

# aws-tag
aws-tag tag \
    --service kds \
    --filter 'team=data' \
    --filter '@name^intel-' \
    --tag 'subteam=intelligence'

Another set of powerful commands are export and import. The export command enables exporting the tags of a subset of resources, which cannot be done simply with AWS Resource Groups Tagging API but it is possible to export tags in AWS Resource Groups service in AWS console. On the other hand, the import command enables to tag any resource with any tag at once. The combination of export and import works harmonically to edit multiple tags of multiple resources with different values, which is very helpful.

aws-tag export --service ebs --filter 'team=data' --file tags.csv

aws-tag import --file tags.csv

Using AWS resource tags for organization, cost allocation, automation and access control use cases are quite common and recomended. However, in order to achieve a successful output, cloud users have to consistently apply same tagging strategies across teams and business units. While preventative policies help to govern the resource tags, using open source tools like aws-tag can streamline the process of tagging previously untagged resources or changing tagging strategies using several rules to achieve the perfect fleet of fully tagged resources.