Detect and block malware with known hashes. Block malicious IP addresses and host names. Create signatures for network packets or sessions. Set policies about allowed and blocked activities such as plugging in USB drives. Have we secured the network yet?
Many security operators have applied a traditional controls-oriented mindset to their strategies. They implement firewall and endpoint rules as safeguards and create company policies to block or detect practices dangerous to security. Every piece of data generated either immediately generates some sort of blocking action, or it might generate an alert for an analyst. That analyst will then determine if the alert was correctly generated or not. Security Operations Centers may use a Security Information and Event Management system to view these alerts, but the alerts are not correlated, just shown in a single pane of glass.
While these organizations continue with very simple detections, the hackers are not so complacent. They find ways to circumvent safeguards and make their traffic look as much like regular traffic as possible. They know the traditional safeguards and know just as easily how to circumvent them. Well, do we just create better safeguards? Do we put more devices, more endpoints, additional policies, until we cripple the network? No.
While machine learning in security has become a hot topic, the trend says something like, “well we use our traditional system and some machine learning on top.” We need to take a step back. What is security operations? We are trying to detect threats and, hopefully, stop or limit their damage as quickly as possible. In other words, we are hoping that threats will have some sort of discernable characteristics which enable us to determine that they are threats and what they are doing. This is a data problem. We collect, query, and/or search data in our system and classify that data or objects generated from that data.
Some in the security operations community may want to think of data engineering, data science, and data analysis as “nice to have skills” in addition to “core security expertise.” In reality, we should think of security operations as a specialization of the data industry. While some problems such as “block bad IP,” are extremely primitive data problem, just seeing if the IP address fields match, they are data problems nonetheless. For a long time, we were limited to these problems due to the challenges or expenses of larger data processing.
Now that true large scale data engineering and analysis is available to more organizations and industries, it is time to start moving up the pyramid of pain. We can only do this if we rethink our overall strategy and have the humility to realize that non-security data-driven organizations may have some of the answers we are looking for.