Simple Ways of Optimizing Your AWS Cost & Workload
All roads lead to Rome or to a cost optimization nowadays.
In this article, 4 ways of improving your existing EC2, Lambda and static content serving workload and cost optimization will be discussed.
TL;DR;
- We’ve reduced the cost of a single project by 18% by separating the static content from EC2 to S3.
- Achieved up to 84% cost decrease in S3 and 62% cost decrease in Lambda respectively.
- AWS Lambda is a panacea for on-demand resources.
- Always re-check your Lambda functions for cost efficiency.
Well, now in order:
- Using S3 for static content serving.
Let’s assume, you have EC2 servers, running some application code, which serves some static JS/HTML/CSS files. Those files are generated only upon a change, which is not necessarily done with the deployment of the main code. The files can be stored in EBS, Elastic Block Storage, attached to the EC2 instance or in the instance itself.
In other words, you have an application running on EC2 that uses some static helpers, which can be deployed independently.
Your graph will be as follows:
What if you derive the static part from your application and deploy independently? It will save you in terms of:
- Server cost
- Server maintaining
Let’s look at the benefits one by one.
1.1 Cost
S3 is cheaper than traditional EBS or even storing in EC2. Let’s assume, you’re using EBS. S3 Standard Tier starts with $ 0.023 per GB, compared to $ 0.08 of EBS per month (General Purpose), where you need to pay for the provisioned capacity. Here’s a full list of pricing for S3 and EBS.
Moreover, S3 can hold a larger amount of data than EBS, but in this article we’ll be speaking of the static content, which is usually not that heavy.
1.2 Server maintaining
EBS is maintained by the application owner, you will need to change the volume configurations, if needed. Though it can be done with a few clicks, S3 lets you store the objects without thinking of the capacity or performance, no manual interference needed.
Moreover, EBS can only be used with EC2 instances, while S3 can be used by most of AWS servers or accessed by multiple instances at a time, which allows us more flexibility.
1.3. Version control
In addition to the advantages of S3, here comes the versioning control, which you can enable for your bucket and retrieve the older version, if necessary.
So, with moving the static content to the S3, your graph will become as follows:
Same JS/CSS/HTML content will be served to the user, but from S3, with the same availability of 99.99%, as it is with EBS. At Insider, by moving the static content of one single project, we managed to reduce the cost of the EC2 instances to 18%.
2. Revise your architecture and adjust according to current needs.
When you first design your product, you can not always predict the best fit and might change or adjust your architecture later. At Insider, one of our Lambda functions gets data from database, converts it to a specific format and writes data in S3, which is later retrieved by end-users on websites:
During the first years of the product release, we were writing all possible data from DB to S3 on every generate request, which was not causing trouble in terms of CPU and cost.
Later, with wider range of customers all across the globe, we needed to implement a solution, which would both
- decrease runtime and memory cost for Lambda
- decrease the S3 billing.
The solution we came up with is that, instead of writing all possible data, derive only the ones, which were modified.
An important aspect I need to mention is that, S3 is billed per requests and so does a Lambda for CPU and time consumed (we’ll cover this part in section 4). As a result, from getting only altered data;
- Lambda function was working with less data, so was faster and consumed less memory
- S3 PUT requests number, the amount of saves to S3 from Lambda, has significantly decreased, which also reduced cost.
Paraphrasing the same in numbers, the result were
- 62% drop in 1 Lambda cost
- 84% drop in 1 S3 bucket cost
Impressive, isn’t it?
3. Use serverless when needed.
Nowadays, we all need systems to run servers and also do have to think on provisioning and cost of those. However, in cases like:
- REST API call
- File processing
- Data retrieval and analysis
- Event based actions, etc
where you only need your servers on-demand, using 24/7 running servers or on-demand instances can be costly and can be billed less if you switch to AWS Lambda.
AWS Lambda is a “serverless” service (for us), invoking EC2 instances, which are managed by AWS. It will help your servers in cost and maintenance.
3.1 Server cost
Lambda is charged for:
N number of request * Time (seconds) * Memory (Gb) * monthly compute charges for GB-second + N number of requests * monthly compute charges for price of requests
not for idle time. Only for the amount you have used when the function was running.
For EC2 instances, the billing changes according to the instance types starting from $0.0104 hourly for t3.micro, for example. On-demand instances cost more than spot ones. Full table of pricing is available here.
Making a rough calculation of using t3.micro for only half in a month will result in
360 hours * $0.0104 = $3.74.
With Lambda, assuming you are out of Free Tier with even 100K requests for a month, each having a duration of 3 seconds with even 512 Gb of memory, you will be billed
100,000 number of requests * 3 s * 512 + 100,000 * 0.2 = $2.52
3.2 Server maintaining
Lambdas are all the time accessible, pay-per-usage functions. In other words, you do not need to manage provisioning, yet use the amount of resources you need and make your work done. AWS takes care of the rest.
4. Optimize Lambda cost
As I’ve mentioned earlier, the Lambda is priced for
N number of request * Time (seconds) * Memory (Gb) * monthly compute charges for GB-second + N number of requests * monthly compute charges for price of requests
Where, from the source,
- monthly compute charges for GB-second is $0.0000166667, and
- charges for price of requests are $0.20 per 1M requests;
so are constant, let’s mark them as C and re-write the formula as
N number of request * Time (seconds) * Memory (Gb) * C + N number of requests * C
Number of request is also not predictable and depends on the product usage, let’s mark it as N:
N * Time (seconds) * Memory (Gb) * C + N * C
This leads us to a simplified formula, giving us the idea of Time and Memory variables to be considered.
Theoretically, the more memory you allocate, the faster the execution will be. Try out different memory settings for Memory which lower the cost of each invocation using the above formula to find the best match.
The practice shows that, depending on the function, there will be a minimum and maximum range of memory your function uses. So from your execution graph:
derive a memory setting, letting your function run smoothly and with lesser cost. Section 2 explains a real-case example with 62% benefit.
Hope you enjoyed reading it! 🤞
Open for your comments and suggestions in comments 📝.
And feel free to contact me on LinkedIn as well.
