FinOps: How can you get the most out of DBaaS? — Part I
It’s been four years now since I joined the Cloud Databases team of the ADEO Tech & Data Platform, and we’ve dedicated the past two years to one thing: helping our users in their transition to the cloud.
Originally a bunch of infrastructure DBAs that were automating the installation of packages and tools, we had to build a new offer for the cloud and to help us with that, we focused on managed services, DBaaS (Database As A Service).
Thanks to our new home-made application to deploy and manage DBaaS, we went from 0 databases in the cloud to 900 in a snap.
With such popularity, we were obviously very watchful of our invoices, and how we could get the most from DBaaS and still saving money. This is where FinOps comes in.
FinOps: taking advantage of the cloud model
A definition that I like about FinOps is the one from the foundation of the same name:
It is the practice of bringing financial accountability to the variable spend model of cloud, enabling distributed teams to make business trade-offs between speed, cost, and quality.
Public clouds are fantastic. We’ve built new applications every day, in a very limited short time and with powerful results. But as a great mind said, with great power comes great responsibility. And that responsibility is to monitor your costs, optimize your usages, and always keep that process in mind.
And how to apply this philosophy to database managed services? Well, there’s plenty of options and as I’m sure you like lists as much as I do, let’s look at 7 DBaaS optimization methods.
Optimize your DBaaS usages
N°1: Schedule your environments, and schedule well
You might think that this is an easy one, and you’re right. But it is maybe the most important one.
Let’s take a quick example of a “classic” application with three environments (development, staging, and production) and two kinds of engines (one PostgreSQL on Cloud SQL and one MongoDB on Atlas). This application is used only during business hours, 9 AM to 7 PM for example.
- PostgreSQL cluster on GCP (2 CPU / 8 Gb RAM / 50 Gb SSD) for 24/7 = 109.68 $
- Same configuration but during business hours only = 38.61 $
- MongoDB cluster on GCP (M10) for 24/7 = 85.68 $
- Same configuration but during business hours only = 35.7 $
For 3 different environments, this represents 7 032,96 $ a year against 2 675,16 $ for scheduled DBaaS, a saving of 62%. This is huge, and we can do even better by stopping dev and staging databases during holidays or quiet periods when developers are not working on this project.
So don’t sleep on that, and don’t let your users say “don’t worry, I know how to do a cron job…” because they won’t always do. Here at ADEO, every time a cloud database is deployed, we automatically create two scheduling jobs (stop/start) that users can configure according to their needs, and today we have 1 database out of 3 that is scheduled to stop when it’s useless.
Now that you’re convinced, don’t forget that most DBaaS providers will charge you for every hour that has started. This means that if you are planning to stop your databases for 7 PM, stop them at 6:55 PM and not 7:00 PM!
N°2: Manage your peak loads and follow trends
When we were using on-premises hardware to run our databases, we needed to size our machines to the minimum capacity required by the most important load.
Thanks to managed services, we can now benefit from auto-scaling.
Look at this Postgres cluster:
As we can see, this machine has an average CPU usage of 15/20% during the day, but a very high 70/80% around 1 AM. On an on-premise infrastructure, we would have needed to deploy a machine with a high count of CPU, only for one or two hours.
Don’t do that in the cloud! Most DBaaS providers offer the possibility to dynamically (and sometimes automatically) downscale your services without interruption.
You have an important batch to run at night? Upgrade your database before launching it, and get back to a smaller pattern when it’s done.
Most of your activity is concentrated on business hours? Schedule your down/up-scale according to that.
N°3: Bad designs lead to bad queries which result in inadequate services sizing
When I was young and using on-premises databases, we used to troubleshoot performance issues with the following process: first, we were doing some query analysis, and if, and only if we thought that we’ve reached the maximum capacity of the compute, we were agreeing to upgrade CPU or RAM or anything else.
But guess what, upgrading service has never been easier in the cloud using DBaaS. That’s obviously a great thing, but that’s also a very dangerous option because a natural response to a bottleneck is … upgrading the service.
You should always consider doing a query/schema analysis before upgrading your resources. Databases have not become magic in the cloud, they still need to be tuned to work perfectly. And just like everything else, monitoring and tooling have never been better in the cloud: Prometheus & Datadog integrations (Aiven), Cloud Logging (GCP), Performance Advisors (Atlas), etc…
And DBAs are not dead either! Now that they spend a lot less time installing components or working on infrastructure issues, they are more dedicated to analysis, design conception, queries optimization, and so on. Use them!
N°4: Avoid lift-&-shift, transform your apps!
A common issue we’ve had the last two years is applications that moved their workload directly to the cloud, without any transformation.
“One big cluster on-premise ? One big cluster in the cloud.
No managed service for my engine ? Let’s install RPMs on pure IaaS.”
Ouch. While this could be a mandatory step sometimes to move quickly to the cloud, you should always reconsider your architecture before doing that.
Just like you do not install a complete (and hard to maintain) server to host a small API or a web front/back that can be hosted on serverless products (Cloud Function, App Engine on GCP for example), you might not want to deploy a three nodes cluster for a small amount of data and queries.
Three examples of legacy usages that can be moved effectively to the cloud:
"You have a very large relational database that you use only for analytics purposes. You have 2 To of data, inserting a few Gb more every month, and running some reports on a few Gb too."
Looks like a perfect example of how a solution like BigQuery could very much ease your life and your wallet: flexible charges depending on storage and input volumes, easy to set up and maintain, small redesign of your queries, no license to buy, etc…
"You have a small application that is not storing any important or even persistent data. Actually, you’re using a MySQL server because you’re sharing it with another application or because you had no choice."
Why not use a simple cache solution such as Memory Store or even a small Redis service? And if you have a small amount of configuration to store, maybe you don’t need a database for that, you may use some GCP bucket / S3 storage?
"You are currently using two different database engines for two distinct purposes: Oracle to manage some relational datasets and Couchbase because your developers find it easier to manipulate JSON documents."
Did you know that PostgreSQL has a very powerful JSON extension? And that it is available at DBaaS providers such as Aiven for example? In that way, you’ll be able to move your two use cases inside only one, open-source, database. Afraid of the migration process or performances? This article by Sisense may reassure you.
That’s all, for now, folks, we’ll continue our list in the second part of this article that we’re going to talk about when you should choose wisely your location and take into mind additional costs.
Stay tuned!
Useful links
- FinOps fundation: https://www.finops.org
- FinOps The new operating model for the cloud: https://www.youtube.com/watch?v=VDrcgEne6lU
- Flexera study “State of the Cloud” https://resources.flexera.com/web/pdf/report-state-of-the-cloud-2020.pdf
- Why use FinOps? https://www.apptio.com/blog/why-use-finops/
