A Better Approach to Controlling Modern Data Cloud Costs

  • Shut off unused always-on resources (idle clusters)
  • Leverage spot instance discounts
  • Optimize the configuration of auto-scaling data jobs

Pros and Cons of Existing Cloud Cost Management

ALLOCATING COSTS — OKAY, NOT GREAT

Here’s a cloud budget management dashboard that does a good job of summarizing at a high level where cloud spend is going. But where could costs be brought down?

ELIMINATING IDLE RESOURCES — GOOD

REDUCING COSTS — POOR

A Smarter “Workload-Aware” Approach

PREVENTING OVERSPENDING

Application-level intelligence about actual utilization allows you specify with precision the number and size of instances really needed for the job at hand.
Knowing the duration of each job empowers you to shut down clusters based on execution state rather than waiting for an idle state.

LEVERAGING SPOT INSTANCES

AUTO-SCALING EFFECTIVELY

To take advantage of cost savings with auto-scaling, you have to know exactly when you need max capacity and for how long — at the application level.

Conclusion

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

The Stata Frames Guide

Top 5 Cool Things You Can Do On Amock.io

Encapsulation- Object Oriented Principles in Typescript

The Way of the Rubber Duck 2

Flutter: Firebase GitHub Authentication

TCP 3-way handshake

While Loop with If else condition

Interrupts Handlers

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sara Petrie

Sara Petrie

More from Medium

Big data and the Cloud: A Blog about the Implementation of Big Data in the Cloud

Athena Partition Projection sample queries ddl’s

The Good, The Bad, and The Ugly

Everything You Need to Know about Data Migration