Support for a Greater Quantity of Standing Alerts with IBM Cloud Pak for AIOps 4.5.0
A lot of the focus with an AIOps platform is on the number of events it can ingest and stream through. However, a big consideration should also be placed on the number of active alerts (also referred to as the standing count) that the platform can handle. An enterprise AIOps platform such as IBM Cloud Pak for AIOps supports clients that process from tens to hundreds of millions of events per day. Even when achieving 99% noise reduction through event de-duplication, suppression, several types of correlation, and prioritization algorithms, this still can leave a substantial number of active alerts.
By supporting 200,000 active alerts, IBM Cloud Pak for AIOps now equals or exceeds the standing count in IBM Netcool, OMNIbus ObjectServer, and IBM Netcool Operations Insights while supporting many more analytics operations.
There are 3 areas to consider when looking at the supported active alerts:
1. How is the ingestion of additional events impacted by the standing alerts?
2. How are the analytics being performed impacted by the standing alerts?
3. How are the active alerts shown to a user?
How is the ingestion of additional events impacted by the standing alerts?
A straightforward concern is whether there is any performance impact in ingesting events into an IBM Cloud Pak for AIOps deployment depending on the amount of alerts contained therein. In short, the answer is that it does not. The time spent ingesting events stays constant, regardless of the amount of standing alerts, supporting a steady rate of 700 events per second, with bursts up to 1,000 events per second (for details on the assumptions made, see the Hardware requirements for IBM Cloud Pak for AIOps). In other words, IBM Cloud Pak for AIOps can support over 60M events per day, regardless of the number of standing alerts.
How is any analytics being performed impacted by the standing alerts?
The reason for having active alerts is not only to display them, but also for analytics to reference them. For example, with de-duplication, an incoming event is first matched to existing active alerts and added to them when they match. With alert correlation, a given alert is potentially correlated with any of the active alerts. IBM Cloud Pak for AIOps has been engineered in a way that these analytics operations are applied consistently and in near real-time even with large active alert counts.
How are the active alerts shown to a user?
The third consideration is displaying the alerts to users. Users view alerts in many different ways, with custom filters, search queries, custom attributes, and customized sets of columns in an alert list view. This is a lot more efficient than looking at a single list with 200,000 rows. As such, IBM Cloud Pak for AIOps supports a configurable limit for the number of alerts to be displayed. This limit supports the ability to filter, to search, and to customize the set of attributes and columns displayed. To give a tradeoff between cluster load and the number alerts, limit can be set to up to 10,000 alerts.
To ensure users never miss an alert, the number of alerts for the given custom filters and search queries are always clearly shown. This way a user can always filter further if they have too many alerts to easily handle.
Conclusion
IBM Cloud Pak for AIOps 4.5.0 supports a standing alert count of 200,000 alerts. This support is across its end-to-end workflow, from data ingestion, analytics, and display. This is just a first step as we are continuously working to further grow the scale for IBM Cloud Pak for AIOps.
Learn more about Cloud Pak for AIOps at https://www.ibm.com/products/cloud-pak-for-aiops
A special thanks to Chris Appleton, Matas Bartosevicius, Cormac Cummins, Mary-Jo Linehan-McGuinness, and Joshua Suckling.
