Cost Optimizations With ECS
Introduction
A few months ago, we converted our daily batch processing into a file-based process. By doing so, we went from the safe confines of EMR world into the unknown territory that’s ECS and more specifically, Fargate. As we’re always focused on how we can save money, what strategies can we apply to Fargate to bring down our costs?
Autoscaling
Similar to EMR, autoscaling allows an ECS service to only run however many tasks it needs based on the current processing load. As a result, you want to make sure you’re configuring your services with a lower minimum and desired number of tasks so that you’re not always using more than you necessarily need.
This piece of knowledge came in handy, especially with regards to our lower environments. As those environments are only running when we need to test changes, we want to set the minimum and desired number of tasks to 1 (or even 0) so that the lights are on, but not shining too brightly. We can’t do this in production as that process needs to run around the clock (although batch processes in production could possibly benefit from this approach), but the savings in the lower environments are definitely more than enough to suffice.
Fargate Spot
We all know the great risk/reward that EC2 Spot instances offer when it comes to compute costs, and Fargate Spot isn’t any different. The savings over regular Fargate are wonderful, but it’s possible that a task can be shut down at any point. The good news is that ECS clusters offer a capacity provider strategy where you can specify that a certain amount of your tasks run on regular Fargate while the remainder run on Fargate Spot. This allows you to have higher availability while still getting savings.
We’re currently trying this out in our lower environments to see if it’s possible to bring to production, just like we did with EMR back in the day. So far, we’re running with a mixture of 80% Fargate Spot and 20% Fargate, with no issues to this point. While we do want to test for a while longer before we come to a conclusion, it’s an encouraging sign that Fargate Spot seems available enough to have at such a rate, and perhaps even higher (maybe we’ll go all the way in Dev?).
Graviton
AWS loves to promote their Graviton processors whenever they can, and for good reason. They offer the best price-performance ratio by far, and so it’s good to see that it’s also available with Fargate as well. The big caveat with this, though, is that this is not compatible with Fargate Spot. Therefore, you can’t get that wonderful combo of savings like you can in EC2 (for now; perhaps this can be a possibility in the future).
Conclusion
ECS, depending on the size of your tasks and the frequency of those tasks, can become a costly service very quickly if you’re not careful with your practices. Autoscaling, Fargate Spot, and Graviton are all good ways to keep those costs in check.
