Open in app

Sign in

Write

Sign in

Calculate Price of EBS Volumes with Python

Stefan Roman
5 min readMay 10, 2019

--

This post describes a tool that I have created for calculating monthly price for used and unused EBS volumes per EBS type. The reason for creating this tool is that AWS’ Cost Explorer did not provide necessary information for me to utilise it.

Background

My company has asked me to estimated monthly price of our application in one region, as we are about to deploy the application in another region, and estimated increase in costs was required. We are using mostly EC2 instances, some s3 buckets and lambda functions. The price of s3 storage is basically zero, since our application is not using a lot of it, and lambda functions are not utilised enough to even get a bill for them yet! The only thing left to calculate would be the EC2 instances, which is straight forward even though we use a variety of instance sizes. I found myself suck with EBS volumes, as we use a huge variety of sizes and EBS types for each instance. I quickly realised this will not be that easy.

I have made a Python script that would access AWS API to fetch EBS volumes per type and add them all up so that my calculation would be easier. I realised that I could push this further by querying Pricing API for EBS prices. After some failed tries querying the Pricing API, I have build a script that gives a tabular report of used and unused (unattached) EBS volumes and their monthly price.

katapultcloud/price_of_ebs

This script provides a pricing and capacity summary of EBS volumes in a specified region - katapultcloud/price_of_ebs

github.com

How to Fetch EBS Prices From Pricing API

During my research for this project I have realised that many people (including me) did not know how to get EBS prices from Pricing API. EBS volume prices are hidden under AmazonEC2 pricing!

To get raw results from Pricing API use the following script. It takes region code as an argument e.g. eu-west-1

#!/usr/bin/env python3
import boto3
import pprint
import sysaws_region_map = {
    'ca-central-1': 'Canada (Central)',
    'ap-northeast-3': 'Asia Pacific (Osaka-Local)',
    'us-east-1': 'US East (N. Virginia)',
    'ap-northeast-2': 'Asia Pacific (Seoul)',
    'us-gov-west-1': 'AWS GovCloud (US)',
    'us-east-2': 'US East (Ohio)',
    'ap-northeast-1': 'Asia Pacific (Tokyo)',
    'ap-south-1': 'Asia Pacific (Mumbai)',
    'ap-southeast-2': 'Asia Pacific (Sydney)',
    'ap-southeast-1': 'Asia Pacific (Singapore)',
    'sa-east-1': 'South America (Sao Paulo)',
    'us-west-2': 'US West (Oregon)',
    'eu-west-1': 'EU (Ireland)',
    'eu-west-3': 'EU (Paris)',
    'eu-west-2': 'EU (London)',
    'us-west-1': 'US West (N. California)',
    'eu-central-1': 'EU (Frankfurt)'
    }ebs_name_map = {
    'standard': 'Magnetic',
    'gp2': 'General Purpose',
    'io1': 'Provisioned IOPS',
    'st1': 'Throughput Optimized HDD',
    'sc1': 'Cold HDD'
}region = sys.argv[1]
resolved_region = aws_region_map[region]
aws_pricing_region = "us-east-1"
pricing_auth = boto3.client('pricing', region_name=aws_pricing_region)for ebs_code in ebs_name_map:
    response = pricing_auth.get_products(ServiceCode='AmazonEC2', Filters=[
        {'Type': 'TERM_MATCH', 'Field': 'volumeType', 'Value': ebs_name_map[ebs_code]}, 
        {'Type': 'TERM_MATCH', 'Field': 'location', 'Value': resolved_region}])    pprint.pprint(response)

The output is rather chaotic!

{'FormatVersion': 'aws_v1',
 'PriceList': ['{"product":{"productFamily":"Storage","attributes":{"storageMedia":"HDD-backed","maxThroughputvolume":"40 '
               '- 90 MB/sec","volumeType":"Magnetic","maxIopsvolume":"40 - '
               '200","servicecode":"AmazonEC2","usagetype":"EUC1-EBS:VolumeUsage","locationType":"AWS '
               'Region","location":"EU (Frankfurt)","servicename":"Amazon '
               'Elastic Compute Cloud","maxVolumeSize":"1 '
               'TiB","operation":"","maxIopsBurstPerformance":"Hundreds"},"sku":"WZCW2T5BKNTTAJG9"},"serviceCode":"AmazonEC2","terms":{"OnDemand":{"WZCW2T5BKNTTAJG9.JRTCKXETXF":{"priceDimensions":{"WZCW2T5BKNTTAJG9.JRTCKXETXF.6YS6EN2CT7":{"unit":"GB-Mo","endRange":"Inf","description":"$0.059 '
               'per GB-month of Magnetic provisioned storage - EU '
               '(Frankfurt)","appliesTo":[],"rateCode":"WZCW2T5BKNTTAJG9.JRTCKXETXF.6YS6EN2CT7","beginRange":"0","pricePerUnit":{"USD":"0.0590000000"}}},"sku":"WZCW2T5BKNTTAJG9","effectiveDate":"2018-07-01T00:00:00Z","offerTermCode":"JRTCKXETXF","termAttributes":{}}}},"version":"20180719234605","publicationDate":"2018-07-19T23:46:05Z"}'],
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '1166',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Fri, 20 Jul 2018 18:44:02 GMT',
                                      'x-amzn-requestid': 'df9a89a1-8c4c-11e8-92f9-c54735d924d8'},
                      'HTTPStatusCode': 200,
                      'RequestId': 'df9a89a1-8c4c-11e8-92f9-c54735d924d8',
                      'RetryAttempts': 0}}
{'FormatVersion': 'aws_v1',
 'PriceList': ['{"product":{"productFamily":"Storage","attributes":{"storageMedia":"SSD-backed","maxThroughputvolume":"160 '
               'MB/sec","volumeType":"General '
               'Purpose","maxIopsvolume":"10000","servicecode":"AmazonEC2","usagetype":"EUC1-EBS:VolumeUsage.gp2","locationType":"AWS '
               'Region","location":"EU (Frankfurt)","servicename":"Amazon '
               'Elastic Compute Cloud","maxVolumeSize":"16 '
               'TiB","operation":"","maxIopsBurstPerformance":"3000 for '
               'volumes <= 1 '
               'TiB"},"sku":"WHD78M63XVBREKT5"},"serviceCode":"AmazonEC2","terms":{"OnDemand":{"WHD78M63XVBREKT5.JRTCKXETXF":{"priceDimensions":{"WHD78M63XVBREKT5.JRTCKXETXF.6YS6EN2CT7":{"unit":"GB-Mo","endRange":"Inf","description":"$0.119 '
               'per GB-month of General Purpose SSD (gp2) provisioned storage '
               '- EU '
               '(Frankfurt)","appliesTo":[],"rateCode":"WHD78M63XVBREKT5.JRTCKXETXF.6YS6EN2CT7","beginRange":"0","pricePerUnit":{"USD":"0.1190000000"}}},"sku":"WHD78M63XVBREKT5","effectiveDate":"2018-07-01T00:00:00Z","offerTermCode":"JRTCKXETXF","termAttributes":{}}}},"version":"20180719234605","publicationDate":"2018-07-19T23:46:05Z"}'],
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '1205',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Fri, 20 Jul 2018 18:44:02 GMT',
                                      'x-amzn-requestid': 'dfbb7f22-8c4c-11e8-92f9-c54735d924d8'},
                      'HTTPStatusCode': 200,
                      'RequestId': 'dfbb7f22-8c4c-11e8-92f9-c54735d924d8',
                      'RetryAttempts': 0}}
{'FormatVersion': 'aws_v1',
 'PriceList': ['{"product":{"productFamily":"Storage","attributes":{"storageMedia":"SSD-backed","maxThroughputvolume":"320 '
               'MB/sec","volumeType":"Provisioned '
               'IOPS","maxIopsvolume":"20000","servicecode":"AmazonEC2","usagetype":"EUC1-EBS:VolumeUsage.piops","locationType":"AWS '
               'Region","location":"EU (Frankfurt)","servicename":"Amazon '
               'Elastic Compute Cloud","maxVolumeSize":"16 '
               'TiB","operation":""},"sku":"6EW9RF7RXHYJKH4W"},"serviceCode":"AmazonEC2","terms":{"OnDemand":{"6EW9RF7RXHYJKH4W.JRTCKXETXF":{"priceDimensions":{"6EW9RF7RXHYJKH4W.JRTCKXETXF.6YS6EN2CT7":{"unit":"GB-Mo","endRange":"Inf","description":"$0.149 '
               'per GB-month of Provisioned IOPS SSD (io1) provisioned storage '
               '- EU '
               '(Frankfurt)","appliesTo":[],"rateCode":"6EW9RF7RXHYJKH4W.JRTCKXETXF.6YS6EN2CT7","beginRange":"0","pricePerUnit":{"USD":"0.1490000000"}}},"sku":"6EW9RF7RXHYJKH4W","effectiveDate":"2018-07-01T00:00:00Z","offerTermCode":"JRTCKXETXF","termAttributes":{}}}},"version":"20180719234605","publicationDate":"2018-07-19T23:46:05Z"}'],
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '1151',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Fri, 20 Jul 2018 18:44:03 GMT',
                                      'x-amzn-requestid': 'dfda03a3-8c4c-11e8-92f9-c54735d924d8'},
                      'HTTPStatusCode': 200,
                      'RequestId': 'dfda03a3-8c4c-11e8-92f9-c54735d924d8',
                      'RetryAttempts': 0}}
{'FormatVersion': 'aws_v1',
 'PriceList': ['{"product":{"productFamily":"Storage","attributes":{"storageMedia":"HDD-backed","maxThroughputvolume":"500 '
               'MiB/s","volumeType":"Throughput Optimized '
               'HDD","maxIopsvolume":"500 - based on 1 MiB I/O '
               'size","servicecode":"AmazonEC2","usagetype":"EUC1-EBS:VolumeUsage.st1","locationType":"AWS '
               'Region","location":"EU (Frankfurt)","servicename":"Amazon '
               'Elastic Compute Cloud","maxVolumeSize":"16 '
               'TiB","operation":""},"sku":"MSGXHGPU2XUF77P6"},"serviceCode":"AmazonEC2","terms":{"OnDemand":{"MSGXHGPU2XUF77P6.JRTCKXETXF":{"priceDimensions":{"MSGXHGPU2XUF77P6.JRTCKXETXF.6YS6EN2CT7":{"unit":"GB-Mo","endRange":"Inf","description":"$0.054 '
               'per GB-month of Throughput Optimized HDD (st1) provisioned '
               'storage - EU (F '
               'rankfurt)","appliesTo":[],"rateCode":"MSGXHGPU2XUF77P6.JRTCKXETXF.6YS6EN2CT7","beginRange":"0","pricePerUnit":{"USD":"0.0540000000"}}},"sku":"MSGXHGPU2XUF77P6","effectiveDate":"2018-07-01T00:00:00Z","offerTermCode":"JRTCKXETXF","termAttributes":{}}}},"version":"20180719234605","publicationDate":"2018-07-19T23:46:05Z"}'],
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '1185',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Fri, 20 Jul 2018 18:44:03 GMT',
                                      'x-amzn-requestid': 'dff94b74-8c4c-11e8-92f9-c54735d924d8'},
                      'HTTPStatusCode': 200,
                      'RequestId': 'dff94b74-8c4c-11e8-92f9-c54735d924d8',
                      'RetryAttempts': 0}}
{'FormatVersion': 'aws_v1',
 'PriceList': ['{"product":{"productFamily":"Storage","attributes":{"storageMedia":"HDD-backed","maxThroughputvolume":"250 '
               'MiB/s","volumeType":"Cold HDD","maxIopsvolume":"250 - based on '
               '1 MiB I/O '
               'size","servicecode":"AmazonEC2","usagetype":"EUC1-EBS:VolumeUsage.sc1","locationType":"AWS '
               'Region","location":"EU (Frankfurt)","servicename":"Amazon '
               'Elastic Compute Cloud","maxVolumeSize":"16 '
               'TiB","operation":""},"sku":"ZYSKSVGZYKJ85CXE"},"serviceCode":"AmazonEC2","terms":{"OnDemand":{"ZYSKSVGZYKJ85CXE.JRTCKXETXF":{"priceDimensions":{"ZYSKSVGZYKJ85CXE.JRTCKXETXF.6YS6EN2CT7":{"unit":"GB-Mo","endRange":"Inf","description":"$0.03 '
               'per GB-month of Cold HDD (sc1) provisioned storage - EU '
               '(Frankfurt)","appliesTo":[],"rateCode":"ZYSKSVGZYKJ85CXE.JRTCKXETXF.6YS6EN2CT7","beginRange":"0","pricePerUnit":{"USD":"0.0300000000"}}},"sku":"ZYSKSVGZYKJ85CXE","effectiveDate":"2018-07-01T00:00:00Z","offerTermCode":"JRTCKXETXF","termAttributes":{}}}},"version":"20180719234605","publicationDate":"2018-07-19T23:46:05Z"}'],
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '1151',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Fri, 20 Jul 2018 18:44:03 GMT',
                                      'x-amzn-requestid': 'e0184525-8c4c-11e8-92f9-c54735d924d8'},
                      'HTTPStatusCode': 200,
                      'RequestId': 'e0184525-8c4c-11e8-92f9-c54735d924d8',
                      'RetryAttempts': 0}}

The following Python script queries Pricing API and extracts prices from the chaotic JSON!

#!/usr/bin/env python3
import boto3
import pprint
import json
import sysaws_region_map = {
    'ca-central-1': 'Canada (Central)',
    'ap-northeast-3': 'Asia Pacific (Osaka-Local)',
    'us-east-1': 'US East (N. Virginia)',
    'ap-northeast-2': 'Asia Pacific (Seoul)',
    'us-gov-west-1': 'AWS GovCloud (US)',
    'us-east-2': 'US East (Ohio)',
    'ap-northeast-1': 'Asia Pacific (Tokyo)',
    'ap-south-1': 'Asia Pacific (Mumbai)',
    'ap-southeast-2': 'Asia Pacific (Sydney)',
    'ap-southeast-1': 'Asia Pacific (Singapore)',
    'sa-east-1': 'South America (Sao Paulo)',
    'us-west-2': 'US West (Oregon)',
    'eu-west-1': 'EU (Ireland)',
    'eu-west-3': 'EU (Paris)',
    'eu-west-2': 'EU (London)',
    'us-west-1': 'US West (N. California)',
    'eu-central-1': 'EU (Frankfurt)'
    }ebs_name_map = {
    'standard': 'Magnetic',
    'gp2': 'General Purpose',
    'io1': 'Provisioned IOPS',
    'st1': 'Throughput Optimized HDD',
    'sc1': 'Cold HDD'
}region = sys.argv[1]
resolved_region = aws_region_map[region]
aws_pricing_region = "us-east-1"
pricing_auth = boto3.client('pricing', region_name=aws_pricing_region)for ebs_code in ebs_name_map:
    response = pricing_auth.get_products(ServiceCode='AmazonEC2', Filters=[
        {'Type': 'TERM_MATCH', 'Field': 'volumeType', 'Value': ebs_name_map[ebs_code]}, 
        {'Type': 'TERM_MATCH', 'Field': 'location', 'Value': resolved_region}])
    for result in response['PriceList']:
        json_result = json.loads(result)
        for json_result_level_1 in json_result['terms']['OnDemand'].values():
            for json_result_level_2 in json_result_level_1['priceDimensions'].values():
                for price_value in json_result_level_2['pricePerUnit'].values():
                    continue
    ebs_name_map[ebs_code] = float(price_value)pprint.pprint(ebs_name_map, width=5)

Output is what you can work with in your projects!

{
 'gp2': 0.119,
 'io1': 0.149,
 'sc1': 0.03,
 'st1': 0.054,
 'standard': 0.059
}
AWS
Price
Ebs
Python
Aws Ebs

--

--

Stefan Roman

Written by Stefan Roman

97 Followers

DevOps engineer who loves Amazon Web Services cloud, automation using available IAC and configuration management tools, and writing own automation tools.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams