Business benefits from AIOps tools

or why they won’t fire a good sysadmin

IT staff stares at the screens and performance metrics of their IT assets around the clock — a typical work shift in any SOC (Security Operational Center) or NOC (Network Operational Center). The resources of your talented engineers can be used much more efficiently by implementing an intelligent system that can handle these tasks on its own. Engineers can be assigned to do work where a person and his intellect are more needed — for example, to tasks of development and control — and routine work should be done by “robots”. This goal can be achieved with the help of an AIOps system but such systems are faced, unfortunately, with a lot of doubts.

How to calculate your AIOps ROI?

The cost is clear — the price of the product and the effort involved in deploying it, but will there be a profit? We will talk about this in this article which will be useful primarily for CDTO, CDO, CIO, who consider AIOps as a promising direction for IT development in their company. It will also be useful for system administrators, SREs and DevOps who are not afraid of changes but make use of them for their own purposes.

As a vendor of MONQ AIOps solutions, we often face questions like “Why change something if it already works?”, “Why do we need one more IT monitoring?”, “We are not ready for this yet”, “It’s expensive”, etc.

The answer will definitely please the CFO. Implementation of AIOps and accompanying changes in the company functioning will lead to higher sales and higher operating margins. Implementing AIOps is an IT investment that has a large positive ROI.

Why is it difficult to start implementing AIOps?

A poll from OpsRamp (“Top Trends In AIOps Adoption: The Future of Digital Operations Management?”) explains why businesses are a little wary of AIOps:

• There is a perception that AIOps tools are expensive to implement, with over 50% respondents indicating that price is a key issue.
• The time-to-value ratio (complexity of deployment) and the current technological limitations of the stack of AIOps tools create ideological challenges within organizations.

Implementing AIOps: stop factors in the minds of IT leaders

The first issue actually exists. But, as they say, “There are different types of AIOps”. Implementing AIOps from CA (Broadcom) or Microfocus starts at $300 per host. If you are a large business with over 100,000 hosts, the cost of such an AIOps is really high. But there are completely different systems, for example, the cloud BigPanda, Moogsoft, LoomSystem or our system, where AIOps implementation with licenses will rarely cost more than $30–50 thousand, simply because we calculate the cost based on the amount of data processed. That is, the client pays only for what he actually uses.

The second issue is the need for organizational change. Today in many large companies IT infrastructure support looks like this:

• Each product or piece of infrastructure has a support team that works in shifts.
• Some employees monitor the status of servers, for example, on a Zabbix screen.
• Another part of the employees watches the pods on the kubernetes in the productive environment.
• Another part of the team monitors user activity graphs and analyzes user complaints about poor performance.
• The process is usually reactive: something broke -> users started swearing -> the support team started investigating -> the problem got fixed.
• The “monopoly” on data is held by the leader of the infrastructure or product support team.

With an AIOps platform and its hybrid (umbrella) monitoring features, the support process would look like this:

• All data from users, logs, and monitoring systems are automatically collected by AIOps without the participation of engineers.
• The system itself determines the level of severity, automatically selects those responsible and launches an auto healing chain, if possible.
• The process is organized proactively: the system continuously monitors, analyzes the state of IT infrastructure and services from the user’s perspective -> predicts failures and assesses their impact on the business -> in case of standard problems, launches auto healing functions -> in case of non-standard problems, it automatically notifies the responsible teams indicating the root cause of the failure.
• Access rights to data can be flexibly configured.

Thus, IT infrastructure support with AIOps looks different, and organizational changes cannot be avoided. If the organizational culture of a company does not include a habit of solving problems quickly, measuring performance by results, and sharing information, then the process of AIOps implementation can be deadlocked.

What are the benefits of AIOps?

There is tremendous interest in the market for the prospects and potential of AIOps. But many people still perceive it as just another “hype”. Predictive analytics can accelerate digital productivity by breaking down operational barriers, eliminating perpetual firefighting, and improving IT efficiency.

In the OpsRamp study respondents expect the following benefits from using AIOps:

• 74% of respondents are expecting opportunities to automate routine operations,
• 67% of respondents believe AIOps can reduce the number and duration of costly service disruptions with faster Mean-Time-To-Resolve (MTTR),
• 58% of IT professionals believe AIOps can help detect anomalies by predicting changes in normal system behavior in dynamic production environments,
• 48% of respondents consider the main profit to be an increase in the speed of finding root causes of accidents (Root Cause Analysis).

Key benefits of implementing AIOps

What is ROI for an IT director?

ROI (Return On Investment) is a coefficient that shows the economic effect (profit or loss) of an investment, in essence, it is a payback ratio. It is calculated using a simple formula:

ROI = (additional income from investment — costs incurred) / costs incurred × 100%

IT support and operations, by their very nature, are costs incurred by any business that relies on IT to work. And more often than not, the CIO justifies certain IT investments by reducing costs, not increasing revenues. And this is the catch.

AIOps should be seen as a multifaceted intelligent machine and a new manufacturing approach. Incorporating it into your processes provides benefits that can and should be articulated in many other ways, not just cost. First of all, these are:

1. Productivity. Due to the fact that a computer can process orders of magnitude more data than a human, and is guaranteed to perform the prescribed operations, for fully automated incidents a reduction in mean time to recovery (MTTR) can reach up to 90%. This is a radical improvement over manual operations. The average increase in labor productivity in our cases is about 60%. That is, the IT department with the same staff of specialists will be able to service twice the infrastructure, or you will be able to redirect them to other important tasks, the level of stress will decrease, and the involvement will increase. One of the key pains of CIOs now — this constant lack of resources — will be eased.

2. Customer service quality. More flexible and adaptive processes directly affect the quality of customer service, especially in retail chains, telecom, electronic services. Here are some facts about the customer experience (Black Morgan, Forbes):

• 96 % of customers believe that customer service plays an important role in their choice of brand loyalty.
• Brands with superior customer experience generate 5.7x more revenue than competitors that lag behind in customer service.
• 77% of consumers say negative customer experiences reduce their quality of life.
• Poor customer service changeover costs US businesses $1.6 Trillion.
• 68% of customers believe a service representative is the key to a positive service experience.
• 90% of executives worldwide, that use data analytics, report that they have improved their ability to deliver a superior customer experience.
• On average, clients tell 9 people about positive experience with the brand, and 16 people about negative experience.
• Companies that passed digital transformation are 26% more profitable than their peers.
• 65% of companies believe that better data analytics is critical to better customer service.
• A 2% increase in customer retention is the same as a cost reduction by 10% with respect to profits.

And what is a positive customer experience? Often this is a well-thought-out pathway for clients and functionality that works like clockwork. Once you understand that AIOps is part of the customer experience improvement process, everything will fall into place.

3. Risk Mitigation. The ability to predict failures, based on historical data from IT incidents, significantly reduces the risk of downtime. For comparison, according to IDC, “the average cost of application downtime for TOP-1000 companies is about $11,000 per minute. That’s about $ 660,000 per hour.”

A company directly profits from cost savings caused by freeing up staff, which may or may not look “amazing” for the business. Next comes the increase in efficiency achieved by eliminating the possibility of human error due to aspects such as skill level, fatigue, stress, etc. But how to extrapolate this increase in efficiency to profits? To do this, the assessment must go beyond the metrics of IT operations and meet the requirements of business operations.

When you view IT operations as a driving force for your business, you need to keep in mind which business entity (line of business), portfolio, or domain you serve. What is the value of your efficiency for such business structures? Is it guaranteed that your distribution channels won’t be blocked, which in turn will improve your customer experience? If so, what value in money do you place on your customer experience? If you have an answer like that this is your return on investment for AIOps.

Why a good sysadmin won’t be left without work (and who else can implement AIOps)

To realize the potential business benefits, a company needs to start changing and adapting to new ways of working before these benefits can be fully realized and quantified.

With the introduction of an AIOps platform, the creation of a permanent implementation team becomes inevitable. A typical implementation team consists of a leader, 2–3 system administrators, a programmer for writing automation scripts, and a business analyst for improving processes. And it is better to be your specialists, and not the contractor’s consultants. AIOps is not a system just to be installed and forgotten, it requires a new approach based on an ongoing process of improvements.

The ideologist of the project should be the CIO or the director of digital transformation, and the main consumers or beneficiaries should be those responsible for the situational center, for monitoring and operation. Those people who yesterday did everything by hand, and today with a press of a button they just start such processes that the computer does most of the work for them. Building a team with clearly defined roles and responsibilities, along with the right management structure, is critical to realizing the potential benefits of AIOps. Once you have such an AIOps team, your IT help desk teams should collaborate with it both for monitoring purposes and for ongoing learning and development. Coping with this change requires focused and empathetic leadership skills. Most importantly: the AIOps platform should be seen as a helper and employee, not an obstacle.

It is better to start gradually, involving more and more new teams in this process, showing through the experience of colleagues from a neighboring unit that everything is possible and you just need to start.

The process of obtaining benefits from AIOps can be illustrated by the following chain:

• Choosing and buying an AIOps platform for a pilot project.
• Determining the pilot area and selecting the implementation group, setting goals and KPIs of the pilot project.
• Transformation of the support team into users of AIOps solutions, connecting data sources, defining correlation rules, setting up dashboards, writing rules and automation scripts.
• Evaluation of the effect of the implemented AIOps pilot on IT operations, including the calculation of changes in MTTD (mean time to discover), MTTR (mean time to response), TCO (total cost of ownership), reducing the number of failures, increasing productivity.
• Scaling the pilot up and implementing AIOps in the organization.
• Evaluation of the effect in terms of operational indicators of reducing the number of customer complaints, increasing NPS, reducing costs, reducing risks.
• Evaluation of the effect on the financial results of the company, including the increase in sales and business margins.

As mentioned earlier, the value of AIOps is multifaceted. In the CIO world, direct benefits will be measured as MTTR, MTTD, and other similar metrics based on alerts and incidents. These performance improvements have a direct and indirect impact on business results such as improved customer service, lower costs, risk aversion, and increased revenue. All of these results can be quantified, and only at this level can the business value of implementing AIOps be measured. But a milestone that quantifies these results can be achieved after implementation of a pilot project. Until then it is just some expenditures for the organization.