Does your cloud bill materialize value or send money into thin air?
Using FinOps to align the variable spend model of cloud to financial accounting principles and deliver value to the business.
The adoption of SaaS tools and cloud platforms is becoming the standard among modern data teams and continues to accelerate (source: Gartner.) However, finance and procurement teams — with traditional, long-established cost-tracking methodologies — struggle to integrate these new usage patterns into their budgets.
Many are the tales of the unhappy data team who executes a successful cloud adoption/migration project, only to come into conflict with Finance after the cloud platform sees organic growth (along with the associated spending increase.)
To their credit, Finance are numbers people — excellent at spotting trends and identifying deviations. So when platform usage shifts from platform migration to platform adoption, and operating costs rise twenty percent from one year to the next, alarm bells sound.
To understand why this happens, one has to recall the traditional state of affairs for the IT procurement department before cloud became a thing.
In days of yore, acquiring tools meant buying a license and installing them — on a desktop, a server, and often, both. Some tools, like databases, often require specialized and dedicated hardware to run on (multiplied by the number of environments [dev, test, prod…]). All told, licenses, hardware, installation, and maintenance, called for a beastly upfront CapEx investment.
But what is CapEx, really?
CapEx vs OpEx, a primer
Many intuitively understand CapEx to mean investment in physical and software tooling and OpEx, the cost of employees and labor. This is true, but the real distinction isn’t in tangible versus intangible, it’s in the value.
Cloud FinOps: Collaborative, Real-Time Cloud Financial Management by J. R. Storment and Mike Fuller explains the difference:
“When you capitalize something, it becomes an asset of the company, whether or not it gets expensed within a specific period. The test you can apply is: if an organization writes a check to acquire something, does that acquisition benefit future periods? If it does, then it can be capitalized. If it benefits only the current period, then it’s an expense that is expended in this period with no future benefit, making it an operational expense. Capitalization causes total outlays to differ from expenses in a similar period, with the delta being that which is capitalized.”
In other words, OpEx expenses, such as employee labor, are directed toward generating value today (in the current period). CapEx expenses, such as hardware and equipment, are paid upfront with the expectation of future benefits.
An alternative to buying assets is renting, which falls under OpEx. But since physical hardware degrades and depreciates over time, renting is unfeasibly costly compared to buying outright.
However, renting a tool as a service, as if it were software, completely changes the math.
More on this in a bit.
Now, back to costs
When investing in on-prem tools and required infrastructure, the up-front costs can amount to millions of dollars. To help absorb the shock of such an investment, organizations would apply Stalin-esque five-year plans (or similar) to spread the damage over the expected lifecycle of the appliance.
While these periodized payments are significant (e.g., $5M expense over five years equates to roughly $83k per month), they have one thing that finance teams love: consistency. You’d pay the same $83k every month (simplified example, not using discounting) for the entirety of the asset’s life. Note, this does not factor in OpEx, which usually flies under the radar as “the cost of doing business.”
But the on-prem cost of “doing business” is not insignificant when you account for all the personnel required to service and maintain the infrastructure. These costs include physical space to house the hardware, networking, security (physical and virtual), backups, failover, patching, and updates, to name a few.
In a previous article, I explained how the true cost of an on-prem warehouse is likely much higher than what you’d pay in cloud, even with variable spend pricing. Using tools like Snowflake’s Business Value Calculator and Artcica’s Snowflake Cost Calculator, you can get an idea of exactly how much you might save.
Separating price from value
“Price is what you pay, value is what you get.”
- Warren Buffet (supposedly)
In the days of yore, much preparation was done in advance of significant CapEx investments in the form of lengthy business value propositions and return on investment projections. Coalitions were formed between IT teams and business users to defend an upfront purchase. Then, a hapless project manager would be selected and thrown to the lions of the project review board, with nothing but her wit and a ppt for defense.
Ironically enough, the scrutiny generally ended there, and little attempt was made to tie the monthly amortized cost to business value. No trivial task, as OpEx expenses are distributed across many CapEx assets (e.g., a single IT team managing servers for the database and web servers). Also, the money spent wasn’t coming back, regardless.
Under such circumstances, it made sense to treat on-prem investment on the basis of accounting rather than value.
However, the pay-as-you-go SaaS model has rekindled finance teams’ interest in tracking utility. If you’re paying as you go, can you prove that you should keep going?
This is not an unreasonable demand; to understand it, we need to go back to defining what we’re paying for: tools.
The utility in utilities
According to Webster’s dictionary, the word “tool” has many meanings (not all of them flattering) so let’s focus on the relevant two:
tool — noun
a) something (such as an instrument or apparatus) used in performing an operation or necessary in the practice of a vocation or profession
b) a means to an end
Tools, in the service of operations, performed or necessary to the practice of a vocation or profession fall into two main categories:
- Enablement: Tools that do things we can’t — I am no more effective at driving nails into boards with my fingers than I am at updating cells by hand across millions of rows. If I want to achieve these ends, I’ll have to shell out for the required equipment.
- Enhancement: Tools that do things we can, better — time is money, so if a washer can wash clothes more efficiently than I can, it frees me up to focus on value-added work, like writing this article.
Using this distinction, we can begin to weigh the cost of a tool against its expected utility toward our business needs.
If an organization needs to analyze data, it needs to comparison-shop for a database that will enable it to do so (an enablement tool). A team of engineers will also be required to load, organize, and glean insights from the data. While no tool can do this entire job, there are tools that augment the team’s efficiency by automating everyday tasks like designing and deploying data models (enhancement tools).
FinOps to the rescue
“The cloud rent is still too damn high,” echoes Finance.
With a deep understanding of the fundamentals, let’s validate that assertion.
To keep things simple, let’s use just two cloud services: Snowflake as an example of a variable spend enablement tool and SqlDBM Database Modeling Tool as a fixed-cost operational enhancement.
To help Finance bring financial accountability to the variable spend model of cloud, a FinOps team has recently been formed, reporting to the CTO (as per FinOps guidelines.)
What did FinOps discover?
Fixed spend enhancement
FinOps starts by scrutinizing SqlDBM, the smaller item on the bill. SqlDBM is a perfect example of a (work/lifestyle) enhancement tool: while it facilitates creating, deploying, and sharing data models collaboratively, it’s not, strictly speaking, required for creating Snowflake tables.
SqlDBM charges $3,000 per user per year (or $250 per month) for a Business license. This sounds like a lot of money, and it may be if we are not getting $3,000 of value from it. So how can we tell?
In a recent interview, Vinoth Kumar, IT Security Consultant at Sophos, a leader in cloud cybersecurity and SqlDBM client, said:
“SqlDBM saves our team thirty to forty hours a month.”
I’m reasonably sure that Sophos consultants are paid a fair market wage, but since I don’t have the data, I can only say for certain that they must make minimum wage or higher. Taking the lower bound of the time estimate means SqlDBM saves Sophos consultants 30 hours of work at $20 per hour per month, which is $600 (or more) — a no-brainer for $250.
Beyond saving money, the actual value in a hosted, fully-managed tool like SqlDBM is in making modeling easy and letting IT security consultants focus on Sophos’ core competitive advantage: IT security.
But the urgent email from Finance wasn’t about SqlDBM. It was about a 20% jump in cloud costs from the previous period — and a twenty percent jump bears investigating.
Variable spend enablement
Let’s turn our attention to Snowflake, an enablement tool that allows the organization to meet its analytic needs and make informed decisions.
It’s important to highlight that the role of FinOps isn’t to challenge spending. Each team has already justified and been granted a budget to meet its analytics needs. FinOps exists to keep those budgets on track by helping distributed teams navigate trade-offs between speed, cost, and quality.
We don’t win by shaving cents from the cloud provider. We win by delivering features to our customers.
Alex Landis, Autodesk
First, the FinOps team built an initial set of dashboards to track cloud consumption in real-time, allowing them to pinpoint consumption spikes in advance of the monthly bill.
Next, the FinOps team engaged the various teams within the organization that use Snowflake (e.g., engineering, warehousing, ML, and business departments) and worked with them to investigate elevated cloud spending as it happened.
Cloud Finops calls this the “Prius Effect,” named after the Toyota Prius’ dashboard that allowed the driver to understand energy consumption and react in real-time. In simple terms:
“Real time reporting + just-in-time processes + teams working together = FinOps”
Thanks to Snowflake’s Resource Monitors, the FinOps team could track (and proactively limit) the credit consumption for individual teams and warehouses. Real-time alerting allowed them to get ahead of the ball by reaching out to the ETL, ML, and Product teams as soon as increased consumption was detected.
Conversations with the ETL team revealed that many new data sources had been added to the daily loads at the request of the business teams. Yes, the engineers followed Snowflake’s best practice recommendations for sizing and keeping the files to 100–250 MB. Spending increase: justified.
Next, the ML team — who were happy to report that the proof-of-concept from the last period was a success and that current costs scaled linearly with the size of the complete dataset being used. Nothing to see here.
This brings us to the Product team. Although their consumption is the smallest of the three and may have gone unnoticed if we were not following FinOps best practices, interrogating their query history helped nip a potential disaster in the bud.
As it happens, the Product team did not use a data model to help them identify relationships between data assets. The misunderstanding led to cartesian joins that multiplied the row count exponentially in long-running queries which burned through credits like California wildfires.
Thanks to FinOps dashboards, alerts, and timely intervention, the queries were fixed, the correct figures emerged, and the damage was capped at twenty percent. While not insignificant, the credit waste could have run to truly extravagant amounts if left unchecked.
The (happy) end
Thanks to economies of scale, zero to no maintenance, accessibility, security, and, most importantly, cost, the rapid rise in the adoption of cloud analytics and SaaS tooling is unsurprising. But the cloud service model has changed the game in other meaningful ways as well.
By turning physical hardware into a service, SaaS tools flip the value paradigm from potential future returns on CapEx to real value-in-the-moment OpEx. But, as with all innovations, business and accounting practices struggle to integrate with new ways of working.
FinOps exists to bridge that gap — and help Finance get a handle on the modern BI landscape. Thanks to FinOps, Finance is no longer a siloed procurement team signing off on costs they cannot measure or put into context.
By leveraging the strengths of the same cloud technology it monitors, FinOps can work together with business and finance teams using real-time metrics to help organizations track, justify, and optimize cloud vendor spending and deliver measurable value to the company, today.
