Terraform for Cost Prevention Reporting
Many products on the market today provide a mechanism for visualizing costs of cloud resources. However, a challenge arises with the need to validate and show the preventative spend within their organization. Pipelines and click-ops automation are not optimized the first time they execute. This can lead to overprovisioning or orphaning of resources throughout the lifecycle of an app, leading to inflated costs that could have been prevented if an appropriate policy had been enforced.
Terraform Cloud and Terraform Enterprise are infrastructure provisioning solutions that provide a cost estimation metric and policy enforcement engine.
Terraform cost estimation compares the resource cost against the targeted cloud provider’s generic per-unit costs. This information is captured in the Terraform state file as a metric and can be used for audit and reporting purposes.
In addition, the HashiCorp Sentinel policy engine can be configured as an insurance policy to hedge against security misconfiguration, cloud spend limits, and internal best practice violations.
Think of it as an insurance policy for your cloud provisioning:
What is insurance?
Insurance is a way to manage your risk. When you buy insurance, you purchase protection against unexpected financial losses. The insurance company pays you or someone you choose if something bad happens to you. If you have no insurance and an accident happens, you may be responsible for all related costs.
Note the bolded sections above. This is really where a properly defined Sentinel policy fits. It protects against those unexpected financial losses. Without a policy in place, you run the risk of overspend.
Sentinel can prevent the deployment of resources that violate the policy settings. Preventing the spend is one thing, but how would you provide the metrics to show a cost savings over a set amount of time across your entire subset of workspaces or organization?
So let’s take the scenario that you need to write a cost prevention policy and apply it to your deployment platform. Along with this policy you need to showcase the cost prevention collectively over a period of time. Thereby visualizing your cloud costs and prevent overspend.
But wait, how would you go about doing this?
Did you know?
In addition to acting as a policy enforcement engine, Sentinel can perform API commands such as a POST call to an external target with the information in the Terraform metadata. The Terraform runs gather forecast spend values from the cost estimation engine. Those values are delta monthly cost, prior monthly cost, and the proposed monthly cost of the workspace.
Terraform Run Mock — Cost Estimate Data Examplecost_estimate = {
"delta_monthly_cost": "51.22",
"prior_monthly_cost": "100.0",
"proposed_monthly_cost": "151.22",
}
The above example is exported from the Terraform run mock file
Create an Airtable Base to send exported Terraform Sentinel Run Data
Let’s take that cost data that Terraform creates and do something useful with it. By sending the data to an external source, it can be evaluated and manipulated by business intelligence (BI) software. The tool of choice for this demonstration is a product called Airtable.
Airtable advertises itself as a low-code platform for building collaborative apps. I refer to it as Excel on steroids. The data in Airtable can be manipulated and transformed into an Interface Dashboard.
Below is a link to the public example that is showcased in this blog. The base table column names (such as run_id, Organization, Workspace Name, etc) directly correlate to the Sentinel Policy API calls that will be discussed in the next section.
- Feel free to copy the Airtable base to your own Airtable Workspace
Public Invite Link to Airtable Interface
- This link showcases the data from the previous Airtable sample set as a Dashboard (see below)
Configure a Sentinel Policy Set for your workspaces
This Sentinel policy will provide the functionality to export the costing data to the Airtable target base.
Configure your Terraform Policy set to your needs for the target parameters.
- The total_cost parameter can be configured to the target max cost per workspace.
- The url_path parameter requires the target Airtable base id and table id (https://api.airtable.com/v0/{{base_id}}/{{table_id}}).
- Finally, attain your Airtable API key by following this guide and insert the value for the api_key parameter
Multiple policy sets can be configured if there are different cost metrics required for different workspaces as the policy sets themselves can be configured individually on a per workspace setting.
If you are unfamiliar with configuring Sentinel Policy Sets you can learn more by clicking here
Once you have this policy set configured and assigned to the target workspace(s), perform a Plan and Apply to confirm data is being exported to the target Airtable base table.
Important Note: The Sentinel Policy will not record speculative run data as the intention is to export policy results from Plan and Apply executions only.
Navigate to Airtable and you’ll be able to create killer dashboards as well as collectively view all exported results in a single pane of glass.
Conclusion
Terraform run executions contain other data sets that could be exported for reporting and analytical use. The complete documentation is listed here. Try forking the sentinel repository and making your own POST calls to your Business Intelligent platform of choice!!
https://developer.hashicorp.com/terraform/cloud-docs/policy-enforcement/sentinel/mock
Happy Terraforming!!
Sentinel policies for Terraform Cloud are available only for the Teams & Government and above Business Plans as well as Self-Hosted Enterprise Cloud.
You can sign up for a free trial of Terraform Cloud by clicking the link below
Reference Links
Sentinel GitHub Repository — https://github.com/spiyacy/sentinel_to_airtable
Airtable Demo Base — https://airtable.com/shrTh7TP7Y1twy56M
Airtable Demo Interface — https://airtable.com/invite/l?inviteId=invRGLEMmUOqlJ4xo&inviteToken=c747973b516fc3124acce5169a4163147129f17a713b32c390f0052adac088e8&utm_medium=email&utm_source=product_team&utm_content=transactional-alerts
