Cloud Costs Optimization using INFRACOST

Published in

Version 1

5 min readJan 10, 2024

In today’s world, most applications are cloud-based, offering scalability, efficiency, and flexibility. Managing the costs for these resources or services in the cloud becomes a significant challenge, where the resources not being used efficiently will result in a spike in total cost overnight. Also managing infrastructure efficiently is very crucial. And it becomes easy when Terraform is used, an Infrastructure as a Code (IaC) tool that helps provision and manage the infrastructure in a declarative manner.

In this blog, we will explore how Infracost, Terraform, and Azure DevOps YAML pipelines can be combined to optimize costs and streamline the process.

What is Infracost?

For those who are new to Infracost, it is a robust tool that enables a shift-left approach for cloud costs by providing cost estimates for Terraform before deployment. This allows developers and the DevOps Team to estimate and track costs associated with the infrastructure changes. With Infracost, we can gain visibility into the financial implications of our Terraform code before applying them, enabling us to make decisions before resource provisioning. It supports popular cloud providers like Azure, AWS, and GCP. Click here to learn more about the tool.

We can use the Infracost CLI in a Script YAML task or the extension available in Azure DevOps [Marketplace], but I will explain the Infracost usage as a customized task in YAML.

Pre-Requisites:

An Azure account (you can use a free trial. If you don’t have one, you can create an account for free)
Azure DevOps (If you need help getting started, see Create one)
Terraform code within Azure Repo
Familiarity with Azure DevOps tasks (YAML-based)
Service Connection in Azure DevOps (that connects to Azure account)
Sign up to Infracost (Sign-up)
Infracost API Key

Getting Infracost API Key

To get the Infracost API Key, log in to your Infracost account, and navigate to Org Settings on the top bar next to your profile, API Key can be found under the General tab, copy and add it as a value to the “INFRACOST_API_KEY” variable within the Variable groups of Library in Azure DevOps.

Customizing Tasks in Azure DevOps YAML

To integrate Infracost within the Azure DevOps YAML pipelines, we can create a script task that executes the Infracost CLI. I will be using bash whereas we can also use PowerShell Core. We can include this task before the terraform plan.

Installing Infracost

The below script task downloads and installs Infracost using its installation script from the specified GitHub URL. The environment variables are set to configure the behavior of dotnet CLI during the installation process.

- script: |
      # Install Infracost
      curl -sSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh
    displayName: 'Install Infracost'
    env:
      DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
      DOTNET_CLI_TELEMETRY_OPTOUT: 1

Executing Infracost

The below script task is responsible for running the Infracost analysis on the Terraform code. It extracts the Infracost API key from the environment variable and then executes the infracost breakdown. The primary function of the infracost breakdown command is to analyze the terraform code and provide a detailed breakdown of estimated costs. It considers factors such as resource configurations, service pricing, and usage information to calculate the potential costs associated with each resource. The infracost breakdown command generates an output in JSON format when we point Infracost to a Terraform directory using the --path flag.

--path specifies the path to the Terraform code directory.

--show-skipped specifies the resources that are skipped where Infracost considers non-billable or has limitations in costing.

  - script: |
      infracostApiKey=$(echo "$(INFRACOST_API_KEY)" | tr -d '\r')
      infracost breakdown --path $(Build.SourcesDirectory)/. --show-skipped
    displayName: 'Run Infracost'

NOTE: We need to pass Infracost API Key as an “INFRACOST_API_KEY” variable within the Variable groups of Library in Azure DevOps and link this Variable group to the respective pipeline.

Leveraging Cost Insights

By incorporating Infracost within the Azure DevOps YAML Pipelines, we gain valuable insights into the cost of the resources that will be provisioned/modified. Below is the sample output of the cost breakdown when the Pipeline is run. This output not only provides a quick overview of the estimated costs but also guides approvals on actionable steps for further optimization and understanding of the infrastructure’s financial aspects. Based on the output, we can implement additional steps within the pipeline where we might set up checks for overspeeding against a predefined budget or notify relevant stakeholders.

Summary of Output:

Overall Detected Resources: 13
Estimated Resources: 9(3 with usage-based costs)
Free Resources: 3(eg. azurerm_key_vault, azurerm_resource_group, etc)
Not supported yet: 1 (azurerm_storage_table)

Explore Infracost Usage Profiles for configuring usage-based costs.

Also, check the list of unsupported resources to stay informed about upcoming Infracost updates.

Project: MyADOV1/Test Platform/IAC

 Name                                            Monthly Qty  Unit                      Monthly Cost 
                                                                                                     
 azurerm_app_service_plan.plan                                                                       
 └─ Instance usage (S1)                                  730  hours                           $91.25 
                                                                                                     
 azurerm_application_insights.appi                                                                   
 └─ Data ingested                         Monthly cost depends on usage: $2.30 per GB                
                                                                                                     
 azurerm_monitor_action_group.dapx_alert                                                             
 └─ Email notifications (1)               Monthly cost depends on usage: $0.00002 per emails         
                                                                                                     
 azurerm_storage_account.app_sa                                                                      
 ├─ Capacity                              Monthly cost depends on usage: $0.0192 per GB              
 ├─ Write operations                      Monthly cost depends on usage: $0.059 per 10k operations   
 ├─ List and create container operations  Monthly cost depends on usage: $0.059 per 10k operations   
 ├─ Read operations                       Monthly cost depends on usage: $0.0047 per 10k operations  
 ├─ All other operations                  Monthly cost depends on usage: $0.0047 per 10k operations  
 └─ Blob index                            Monthly cost depends on usage: $0.0376 per 10k tags        
                                                                                                     
 OVERALL TOTAL                                                                                $91.25 
──────────────────────────────────
13 cloud resources were detected:
∙ 9 were estimated, 3 of which include usage-based costs, see https://infracost.io/usage-file
∙ 3 were free:
  ∙ 1 x azurerm_key_vault
  ∙ 1 x azurerm_key_vault_secret
  ∙ 1 x azurerm_resource_group
∙ 1 is not supported yet, see https://infracost.io/requested-resources:
  ∙ 1 x azurerm_storage_table

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project                                            ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ MyADOV1/Test Platform/IAC                          ┃ $91          ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛

Finishing: Run Infracost

Output of Azure DevOps Pipeline:

Conclusion

It’s worth considering the Infracost for its ability to provide detailed insights into the resource costs, enabling the individuals to make informed decisions before provisioning.

Use Cases:

Cost control - Offering a detailed breakdown of resource costs, allowing users to identify and optimize resources.
Decision making - Providing insights into potential costs before provisioning or applying changes.
Continuous Optimization - Integrating Infracost into CI-CD pipelines for ongoing cost management.

Limitations:

Incomplete resource support - Infracost does not support all types of resources and there are few exceptions.
Usage costs - While Infracost helps in analyzing resource costs, but does not factor in usage costs.

About the Author:

Raghavendra BN is a DevOps Engineer, currently working in Version 1’s UKDDC practice. Follow Version 1 and Raghavendra BN for more blogs around Microsoft Azure and Azure DevOps.