Azure Sentinel FUSION: machine learning for a SecOps world

The annual RSA conference just wrapped up in San Francisco. With the introductions of Chronicle’s Backstory (Google) and Azure Sentinel, 2019 became the year of the ‘Cloud SIEM’.

Why is this important? VisibleRisk summarizes it as: “because these types of products can flip two decades of “normal” on their head and finally position those who defend our enterprises in a way that they can keep pace with the furious pace of change they face.”

Azure Sentinel leverages the immense compute power of the cloud and sophisticated machine learning models to help defenses in the enterprise. Microsoft calls is Azure Sentinel FUSION.

Image for post
Image for post

Azure Sentinel FUSION? Say what?

If you go to the Overview page in Azure Sentinel you’ll see a reference in the bottom right corner a section called: Democratize ML for your SecOps. It says:

If you click on the Learn More link it brings you to this page.

Image for post
Image for post

Enabling Fusion

There is no UI to enable Fusion, however if you have an instance of Azure Sentinel running, you can use Azure Cloud Shell and the ‘az’ command to enable Fusion for your Log Analytics workspace.

Start Azure Cloud Shell:

Image for post
Image for post

Run the following command:

az resource update — ids /subscriptions/{Subscription Guid}/resourceGroups/{Log analytics resource Group Name}/providers/Microsoft.OperationalInsights/workspaces/{Log analytics workspace Name}/providers/Microsoft.SecurityInsights/settings/Fusion — api-version 2019–01–01-preview — set properties.IsEnabled=true — subscription “{Subscription Guid}”
Image for post
Image for post

You get back the result that you are now enabled for FUSION:

Image for post
Image for post

OK, now what?

Great question.

Because there is only one page of documentation online, I reached out to the Azure Sentinel product engineering team in Israel, and asked them what Fusion does and got this response:

“Fusion looks at alerts coming from different sources and tries to find out if there’s a connection between them in order to fuse them into one case with higher confidence.”

“Think about having multiple low fidelity alerts that no one had the time to investigate, we tell you if you should investigate them by fusing them into one case.”

Microsoft’s documentation does give another couple of clues:

“Machine Learning in Azure Sentinel is built-in right from the beginning. We have thoughtfully designed the system with ML innovations aimed to make security analysts, security data scientists and engineers productive. One such innovation is Azure Sentinel Fusion built especially to reduce alert fatigue.”

“Fusion uses graph powered machine learning algorithms to correlate between millions of lower fidelity anomalous activities from different products such as Azure AD Identity Protection, and Microsoft Cloud App Security, to combine them into a manageable number of interesting security cases.”

Unified SecOps

Not coincidently, Microsoft announced last week that they are integrating Cloud App Security, Azure ATP and Azure AD identity protection into an unified SecOps experience and portal:

“Microsoft has three identity-centric security products offering detection capabilities across on-premise and in the cloud:* Azure Advanced Threat Protection (Azure ATP) identifies on-premises attacks* Azure Active Directory Identity Protection (Azure AD Identity Protection) detects and proactively prevents user and sign-in risks to identities in the cloud* Microsoft Cloud App Security (MCAS) identifies attacks within a cloud session, covering not only Microsoft products but also third-party applicationsWe are happy to announce that we have brought these together in a unified SecOps experience, which focuses on identity-based alerts and activities for true hybrid identity threat protection.”
Image for post
Image for post

Based on three pillars

So why are all security vendors adding machine learning and artificial intelligence to their solution? Well, first of all: sifting through tons of alerts in a SIEM is not something security analysts love doing. Their skill set can also be better put to work to hunt for bad actors, based on pre-filtered signals.

Secondly, it is well known that security analysts are drowning in those alerts and sometimes miss the critical piece to launch to the next step of investigation. In fact, Mark Russinovich laid out Microsoft’s strategy dealing with this three years ago.

Image for post
Image for post

Ram Shankar, who works on the Microsoft Azure team, wrote that the ML team behind Azure Sentinel FUSION asked three questions:

1) Why are alerts noisy?2) How do experienced security analysts deal with this?3) How can we incorporate domain knowledge into the system?

The ML team came up with these three ideas:

1: Probabilistic Kill Chain

Garden variety detections assume static kill chain. Not true — real world attacks are complex and multistage. So, the ML Team modeled the probability of moving to the next step is conditioned not only on previous step but also factors like current asset.

2: Iterative attack simulation

A lot of noise looks like legit attacks because detections explore only one line of attack. For every alert, the ML team iteratively simulates multiple lines of attack using random walk style algorithms to evaluate if this attack is truly feasible.

3: Encode domain knowledge as priors!

Incorporating Bayesian methods to tap into expert’s domain knowledge is painfully obvious but the common hurdle inference style algorithms are slow. Not a problem because Azure Sentinel is a cloud based SIEM and the ML team can leverage the cloud’s scalable + compute.

These three ideas form the bedrock of Fusion, that Ram claims has shown to reduce alert fatigue by 90%.

Image for post
Image for post

MCAS & Azure ATP

Going back to the Data Collection page in Azure Sentinel and clicking on Azure Advanced Threat Protection (ATP) data source, we find another clue:

Image for post
Image for post

PRO TIP: Both Cloud App Security (MCAS) and Azure Active Directory data sources need to be connected for the current (preview) release of Azure Sentinel Fusion to work.

Azure Sentinel FUSION in action

The scenario we’ll be demonstrating is where a user’s credentials are stolen, and the following actions happen afterwards:

Azure AD identity protection· The user account signs in to an unusual location.
Cloud App Security
· The user’s mailbox gets a suspicious inbox forwarding rule.

Normally these two alerts are seen in different portals and it would take a security engineer to ‘connect the dots’.

However, when you connect these data sources (Azure AD, Azure ATP and Cloud App Security) to Azure Sentinel, the machine learning models behind Azure Sentinel FUSION kick in and generate a Case, showing that data is being exfiltrated:

Image for post
Image for post

In cybersecurity, it’s AI vs. AI

Paul Gillin of SiliconAngle wrote:

“Artificial intelligence research group OpenAI last month made the unusual announcement: It had built an AI-powered content creation engine so sophisticated that it wouldn’t release the full model to developers.

Anyone who works in cybersecurity immediately knew why. Phishing emails, which try to trick recipients into clicking malicious links, originated 91 percent of all cyberattacks in 2016, according to a study by Cofense Inc. Combining software bots to scrape personal information from social networks and public databases with such a powerful content generation engine could produce much more persuasive phishing emails that might even mimic a certain person’s writing style, according to Nicolas Kseib, lead data scientist at TruSTAR Technology Inc.

The potential result: cybercriminals could launch phishing attacks much faster and on an unprecedented scale.”

AI is a new weapon that some people believe could finally give security professionals a leg up on their adversaries.

Conclusion

Microsoft is beating the other security vendors to the punch, having already added some real machine learning models and AI behind their just released Azure Sentinel cloud SIEM offering.

Azure Sentinel FUSION can help reduce alert fatigue, but more importantly ‘connect the dots’ and provide security analysts with a clear picture of the (potential) threat.

— Maarten Goet, MVP & RD

Microsoft MVP and Microsoft Regional Director.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store