Apache Metron: The Emerging Open Source Platform for Security Operations Centre Management
A cursory study of the top data breaches of last year would list the entire key organizations of the world including Governments, Banks, Retail establishments etc.
Cyber Security is now no longer an IT challenge but a business challenge. More enterprise assets are vulnerable today as digital and IoT ecosystems are based on APIs and cloud-native applications. The challenge from a security perspective is multi-layered and arises also from a lack of a holistic approach that combines security with data governance, audit trails, and quality attributes. Traditional solutions cannot handle these threats. Emerging solutions that incorporate deep learning and real-time analytics into their core security design with a view to insights from analyzing large scale data at a very low latency are the need of the hour
To provide for an integrated approach across disparate security platforms and toolsets like intrusion detection systems, firewalls, antivirus tools etc., enterprises have begun investing in integrated Security Operations Center (SOC) platforms. SOC is a centralized capability designed to handle any and all security incidents across millions of endpoints. The goal is to provide tools for the corporate-wide data collection, data aggregation, threat detection, advanced analytic and workflow capabilities — all from a single area of management. Thus, SOC systems perform a highly essential function as they deal with massive amounts of data streams constantly being generated by many different systems, devices and business applications.
All of this data is then pulled into Security Incident and Event Management (SIEM) tools, which then provide reports from a security alert standpoint. A security analyst then determines if these alerts represent a specific threat or are just harmless.
OpenSOC to Apache Merton
On September 15 2015, companies like Hortonworks, Rackspace, ManTech and B23 initiated a top level Apache project called Apache Merton to foster a vibrant open community to accelerate the development of OpenSOC. Metron’s goal is to create open source Apache project dedicated to providing an extensible and scalable advanced security analytics platform to detect and mitigate security risks in real time.
Metron empowers users by enabling them to process huge volumes of data per second, which improves the quality of malware detection and prevention significantly. When an organization is attacked, Metron users can collaborate with each other using data feeds across the platform in real time.
This not only facilitates enhanced detection of malware campaigns but also makes attacks economically unviable for attackers by forcing them to customize malware for each target.
Apache Merton Core Components:
Metron integrates a variety of open source big data technologies in order to offer a centralized tool for security monitoring and analysis. Metron provides capabilities for log aggregation, full packet capture indexing, storage, advanced behavioral analytics and data enrichment. It also adopts the most current threat which is intelligence information to security telemetry within a single platform.
Metron Architecture can be segregated into 5 key functional areas:
1. A mechanism to capture, store, and normalize any type of security telemetry at extremely high rates.
2. Real-time processing and application of enrichments such as threat intelligence, geo-location, and DNS information to telemetry being collected.
3. Efficient information storage based on how the information will be used:
4. Dashboard interface that gives a security investigator a centralized view of data and alerts passed through the system on one single page.
5. Big data Security analytics using Hadoop, Apache Spark, and Apache Zeppelin
Apache Metron will be a game-changer in Cyber Security Software Platforms within the coming years just like many of the other top level Apache Projects like Hadoop, Apache Spark etc.