Progress in Anomaly Detection part4(Machine Learning)
- PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning(arXiv)
Author : Thorsten Wittkopp, Dominik Scheinert, Philipp Wiesner, Alexander Acker, Odej Kao
Abstract : Due to the complexity of modern IT services, failures can be manifold, occur at any stage, and are hard to detect. For this reason, anomaly detection applied to monitoring data such as logs allows gaining relevant insights to improve IT services steadily and eradicate failures. However, existing anomaly detection methods that provide high accuracy often rely on labeled training data, which are time-consuming to obtain in practice. Therefore, we propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows provided by monitoring systems instead of labeled data. Our attention-based model uses a novel objective function for weak supervision deep learning that accounts for imbalanced data and applies an iterative learning strategy for positive and unknown samples (PU learning) to identify anomalous logs. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets and detects anomalous log messages with an F1-score of more than 0.99 even within imprecise failure time windows
2.Incentive-weighted Anomaly Detection for False Data Injection Attacks Against Smart Meter Load Profiles (arXiv)
Author : Martin Higgins, Bruce Stephen, David Wallom
Abstract : Spot pricing is often suggested as a method of increasing demand-side flexibility in electrical power load. However, few works have considered the vulnerability of spot pricing to financial fraud via false data injection (FDI) style attacks. In this paper, we consider attacks which aim to alter the consumer load profile to exploit intraday price dips. We examine an anomaly detection protocol for cyber-attacks that seek to leverage spot prices for financial gain. In this way we outline a methodology for detecting attacks on industrial load smart meters. We first create a feature clustering model of the underlying business, segregated by business type. We then use these clusters to create an incentive-weighted anomaly detection protocol for false data attacks against load profiles. This clustering-based methodology incorporates both the load profile and spot pricing considerations for the detection of injected load profiles. To reduce false positives, we model incentive-based detection, which includes knowledge of spot prices, into the anomaly tracking, enabling the methodology to account for changes in the load profile which are unlikely to be attacks
