Robust Detection of Evasive Malware — Part 1
Malware detection is moving away from hand-crafted rule-based approaches and towards Machine Learning (ML) techniques. In this series of posts, we discuss malware detection with neural networks and how to harden these models against malware variants that are crafted to evade detection. Let’s get started!
Malware Representation
Malware (malicious software) exists in a variety of forms (e.g., PowerShell scripts, PDF files, and binaries). Here, we focus on malware in the form of Portable Executable (PE) files.
Given a PE file, we would like to predict whether it’s malignant or benign. In ML terms, we perform binary classification to label a PE file as of class 0 (benign) or class 1 (malignant). To set an ML pipeline, we need to decide on an appropriate representation of our data points (PE files).
Let’s represent each PE by a binary feature vector indicating its imported functions. In other words, we set an entry of the vector to 1 if the corresponding function is imported by the PE file, and to 0 otherwise. The LIEF tool provides a neat way of doing this as shown below, resulting in a 22761-dimensional feature vector for each PE file.
Note that other established parsing tools can be used (e.g., pefile). Further, more features can be incorporated (e.g., properties of other PE imports). We leave this for future investigation.
Training a Malware Detector
Having represented our data points with binary indicator vectors, we are now ready to build and train an ML-based Malware detector. We used a feed-forward network for our malware classifier with 3 hidden layers of 300 neurons each. The ReLU activation function is applied to all the 3 × 300 hidden neurons. The LogSoftMax function is applied to the output layer’s two neurons which correspond to the two labels at hand: benign and malicious. The model is implemented in PyTorch.
We use 19,000 benign PEs and 19,000 malicious PEs to construct our training (60%), validation (20%), and test (20%) sets. The training set is grouped into mini-batches of 16 PE samples (8 benign, 8 malicious). The classifier’s parameters (denoted by θ in the rest of the post series) are tuned with the negative log likelihood loss, using the ADAM optimization algorithm with a 0.001 learning rate over 150 epochs. We refer to this model by natural since we followed the natural (standard) training pipeline as shown below.
Testing the trained model on 3800 benign and 3800 malicious PEs produced the following performance metrics.
Accuracy: 91.9%
False Positive Rate: 8.2%
False Negative Rate (Evasion Rate): 8.1%
All of this is good to some extent, but what if the malware authors changed the way they import functions to make a malicious file look like a benign one without affecting its malicious functionality. Even worse, malware authors can change the name of the imported functions, and symbols. Let’s see how well our model performs when such adversarial perturbations are applied.
Evasive Malware (Threat Model)
Consider the case where malware authors can import additional functions. That is, the adversarial version of a PE file — represented by its binary feature vector x_adv — obey the following constraint with regard to the PE’s original binary feature vector x.
Pictorially, we have
Given a set of adversarial perturbations, a malware author is interested in the one which evades detection. A brute-force approach would be to try all the possible perturbations. While this is possible, it generally is impractical: for every adversarial version, the author has to unpack the PE, import a couple of functions, pack, and test it against the model.
On the other hand, if the malware author has access to the internals of the model, she could change the binary indicator vector (and the imported functions, respectively) in the direction that maximizes the model’s loss (minimizes its accuracy). From calculus, the gradient points in the direction of steepest ascent. With backpropagation, malware authors can compute the gradient of the model’s loss with respect to their PEs’ binary indicator vectors and uncover the changes (in terms of the additional imported functions) needed to evade the detector.
We overlooked one technicality in our claim above: satisfying the constraints of our model. Constraint I, the resulting feature vector must be binary: it can only take 0 (to denote that the corresponding function is not imported) or 1 (to denote that the corresponding function is imported). Constraint II, the resulting vector must also satisfy x_adv /\ x = x. Stepping along the gradient direction can break one or both the constraints: entries can take arbitrary continuous values. One remedy is to round the perturbed feature vector to the binary domain (Constraint I) and ORing the rounded feature vector with the original binary indicator vector x (Constraint II). We implemented 4 variants of such procedure giving rise to 4 different attack methods, namely rFGSM, dFGSM, BGA, and BCA.
We applied the 4 attack methods on our malicious test set and measured the evasion (false negative) rate of the corresponding generated adversarial versions on the model. From an evasion rate of 8.1%, the model broke down to a 99.7%-evasion rate!!
Attack Method | Evasion Rate (%)
— — — — — — — — — — — —
dFGSM — 99.7
rFGSM — 99.7
BGA — 99.7
BCA — 41.7
This is alarming!! We did not even incorporate the adversarial perturbations where the authors may remove imported functions and replace them with others. What can we do about this? This is the topic of our next post, stay tuned.
