Machine learning in cybersecurity
Owing to technologies which allow for large volume data storage and analysis, companies are able to fulfil tasks which so far have been impossible. Such tasks include widely understood network security, also detecting hazards and attacks.
In the past, such activities were performed only by qualified analysts of network security by monitoring data and creating proper rules for detecting attacks. However, as the size of network data grows, there are more causes for which experts become less and less effective:
1.the growing number of network devices causes that analysts are not able to monitor an increasing quantity of incoming data in the search for hazards;
2.in order to monitor more data, it is necessary to employ more analysts, what in connection with a shortage of qualified personnel is an increasing cost for a company;
3.attacks are becoming more common and cause significant losses — according to Lloyd’s insurance company, they cost USD 400 000 000 000 annually;
4.a delay between detecting a hazard and taking actions, in the event of manual monitoring, is high — until an action is taken, there might already be considerable damage in the system;
5.manually created hazard detection rules skip attacks which do not match such rules.
These problems are now solved by systems based on machine learning.
How does Machine Learning work?
Widely speaking, machine learning refers to a series of techniques where, based on historical data, an algorithm solving a given problem is trained. For instance, detecting cats in photos.
While preparing data correctly (manually tagged photos as “a cat” and “not a cat”) with which machine learning algorithm is supplied, it is possible to create a solution which can effectively detect (classify) whether on new photos coming to the system there are cats. In such a manner, based on historical data, it is possible to anticipate a class of a future incident.
The popular application of machine learning is voice recognition, fraud detecting, anti-spam filters, text processing, product recommendations, video analysis and many others. Machine learning has also been used lately in the field of network security. Since simple tasks are now fulfilled by robots and artificial intelligence, is it possible that only algorithms are responsible for such a complex task as cybersecurity?
This issue is discussed by network experts presenting arguments for and against. At the same time, companies are looking for methods of introducing machine learning as another technology in their anti-cybercrime measures.
Machine Learning in cybersecurity
Currently, the majority of cybersecurity systems based on machine learning is used as a warning system detecting abnormalities in the regular network traffic. As part of this approach, historical data are used in order to learn an algorithm how correct network traffic looks like and label everything what deviates from a standard.
The faster such abnormalities are identified, the faster prospective hazards may be prevented and neutralised.
A main argument against such solutions is too many false alarms which must be analysed by experts. Such an unnecessary activity of analysts frequently results in the drop of trust in the system.
On the other hand, the number of generated network data exceeds the possibilities of analysing by people. The fact that neither experts nor automatic solutions are ideal resulted in solutions where artificial intelligence is aided by professionals.
An example here is the system called AI2 created in CSAIL MIT laboratory. This system analyses dozens of millions of logos every day, separating abnormalities and suspected incidents.
Then, they are handed over to experts which manually label real hazards and reject system mistakes. With time, the system, owing to information from analysts, learns to detect real attacks more effectively, at the same time decreasing the level of false alarms.
Detecting attacks
Owing to such an approach, experts have to analyse about 100–200 incidents a day, whereas earlier those were dozens of thousands of such incidents. A team responsible for AI2 proved that such an approach is capable of detecting 85% attacks with a considerable reduction in the number of false alarms.
Despite these successes, it is too early to tell whether security experts can be replaced by solutions completely basing on artificial intelligence. It may be predicted that in the future computers will play a more and more significant role in this process.
Currently, though, these are the joint efforts of people and machine learning algorithms which bring in the best results against cybercriminals. Yet, it does not mean that there will be a dropping tendency in hazards. Attackers are patient and careful in searching for system loopholes; they create and develop more and more advanced attack methods, also making use of machine learning.
There is a possible scenario for the future that most attacks will be aided by artificial intelligence and protection will consist in creating a better algorithm than a cybercriminal.
Originally published at greywizard.com.
