How to control cloud costs through collaborative BI
When measuring expenditure is not enough
“If you don’t measure something, you can’t manage it!”
This quote by Robert Kaplan, creator of the Balanced Scorecard, reveals a great truth: without being based on certain data it is practically impossible to govern a process and to expect good results from it.
“A system must be managed. It will not manage itself.”
E.W. Deming — The New Economics for Industry, Government, Education (1993)
As you can easily see from an online search, Kaplan and Deming (father of the eponymous cycle called “PDCA Cycle” or Deming Cycle) are not the only ones who think so.
Planning without control does not lead to the best achievable results, bringing also the risk of becoming too strict and therefore unproductive.
In the check phase of Deming cycle, it’s useful to have a BI software with a set of data always available, updated and easily accessible thanks to an adequate graphic representation.
Within the IT department of MailUp Group, which is in charge of managing the cloud procurement and operational management processes, it was necessary to implement a solution for cost control, given the increased share of resources in the cloud.
Applying the logic of continuous improvement to existing processes, the need to control operational activities from a cost point of view was identified, given the Pay as you go nature of the main IaaS providers.
So we thought of a BI tool that is :
- easy to share, with other team or business unit
- free of charge, to avoid paying a tool to save money.
- easily understandable, inspired by the concept in use in Lean Manifacturing “Poka Yoke”.
Choosing the right Business Intelligence tool
The solution was the introduction of software that deals with the reporting of costs generated by our cloud infrastructure.
After some evaluations, it was decided to adopt a tool that would allow the monitoring of costs, a Business Intelligence system.
The choice fell on an open source software called Apache SuperSet, which by definition is available for free.
BI goes collaborative
The above mentioned tool has introduced great improvements, but to really control costs in the cloud is not enough having data placed in dashboards, you need to create a communication channel with those who use the cloud to pass some awareness messages. For example:
- importance of saving
“How would you manage the cloud if the money spent was yours?”
“Can we shut down these resources at night?” - importance of rightsizing
“Is all that computing power really necessary?”
and many, many more messages, helping the creation of a competence centre and the sharing of best practices.
From the point of view of cost optimization, in this case applied to the IT sector, if cost data were extended to the different teams/business units involved, results in terms of awareness and accountability could be achieved.
In fact, it is already possible to hear about “Collaborative Business Intelligence” which seems to offer two main advantages:
- Collaborative decision-making process:
The ultimate goal of any BI program is to make timely and optimal business decisions.
It must also enable organizations to monitor decisions and analyze their validity and impact on business. These analyses provide feedback that can help improve decision making, allowing companies to document best practices and monitor the types of information that provide useful input to the decision making process.
Among other things, it creates a virtuous circle, that is, the increase in collaborative interactions between company managers and workers, sharing the results of the BI and discussing their meaning. - The improvement of information:
Incompleteness of the data automatically supplied to the BI tool is a component that can only be solved by humanizing the data. This is possible in an environment where users can also add their knowledge to the BI results.
Through collaborative processes, workers can share information by correlating data and events. Published BI results can be improved through feedback mechanisms like: evaluations, comments and tags and in some cases, blogs and microblogs.
Wrap up
If this article has helped you or given you an opportunity for improvement that has proved useful, feel free to share your experience with me.
In conclusion, I would like to thank the IT department of MailUp Group for this project, in particular the CIO, IT Procurement and DATA teams, for their support and availability.
