Is your ML Model secure?
Machine Learning (ML) has become almost ubiquitous in solving business problems and create competitive advantage in the marketplace. At the same time, the ML Pipeline and models have evolved into an attack vector and are very attractive for malicious actors.
In this article, we talk about vulnerabilities in common ML models and what measures can be taken to prevent them from corrupting ML models.
Regression is one of the more common types of ML techniques and includes a handful of specific algorithms that are used across the technology industry, such as Linear Regression, Ridge Regression, Logistic Regression, and Lasso Regression, just to name a few. Regression algorithms are used to predict essentially any type of continuous numerical value.
Linear Regression is among the simplest form of regression algorithms, and the goal is to establish a relationship between a dependent value y and one or more independent values x. For example, in order to predict the rating that a user might give a movie (dependent value y), the model uses inputs that include previous viewing behavior and rating behavior of other similar users.
The weakness can occur during training: Because linear regression models are supervised machine learning models (meaning that they learn from a training dataset), attackers can insert malicious input (training data) to cause an availability attack (akin to a Denial of Service attack) or a poisoning integrity attack (where the model’s output is changed to the attacker’s desired output).
This type of attacks can either stop predictions or create bad predictions. Attacks have targeted intrusion detection systems where malicious actors introduced adverse network traffic that stops legitimate traffic causing a denial of service. Anti-spam systems have also been attacked, where malicious actors have crafted custom emails in order to bypass that system and render it ineffective. In these cases where security tools use ML to create better defenses, they have also had to secure against new vulnerabilities introduced with the learning models. This is a lesson for use of ML in critical applications and decisions (e.g. healthcare, infrastructure, finance, etc)
Online learning models are potentially more vulnerable to attacks because the ML model is continuously updated with new training data, providing attackers with more opportunities to insert fake data. In this case, the data science team would need to consider if the model really needs to receive data with such high frequency. If the frequency is lowered to, for example, once per day, it would give enough time to implement processes to validate the data before it enters the learning model.
The field of Adversarial Machine Learning studies these attacks and how to make ML Pipelines more robust to malicious inputs or attacks. As laid out in a new paper by Matthew Jagielski et al., several regression models were tested for robustness using different adversarial defensive models. Among the adversarial models were the Random Sample Consensus (RANSAC) which uses an iterative method to identify outliers, however, attackers can generate inliers (these are data points that appear reasonable to RANSAC, making it difficult to detect. Another adversarial defensive model used was the Reject On Negative Impact (RONI) which was proposed against spam filtering attacks. RONI evaluates model performance including and excluding outliers, depending on the result of the performance data points identified as outliers and not included in the model. Finally, the researchers proposed their own adversarial model TRIM, which calculates the regression parameters iteratively and after few iterations identify the malicious data points that produce a more robust model. In this particular study, TRIM proved to be more effective to test robustness compared with the RANSAC and RONI models.
Adversarial models may be useful to prevent malicious actors from corrupting ML models, so it is worthwhile to evaluate various adversarial defensive models and to adopt the one that best fits your needs. In addition to implementing an adversarial defensive model, other security considerations need to take place as part of data governance and best security practices.
Having clearly defined roles along with role-based access controls (RBACs) can also be helpful. RBACs provide a methodology for regulating user’s access to information assets based on their role within the organization. With a clear role assignment, we also prevent malicious actors from corrupting ML models and their outcomes.
And never forget that security in machine learning is a group effort! Machine learning professionals in conjunction with their security and risk counterparts can work together to improve their security posture. Data Governance is always an extremely important consideration to help decide what data will be included in the machine learning process.
Security in machine learning is a living process. In order to ensure ML model integrity, security needs to be injected into the ML Pipeline using adversarial defensive models and further security measures that enforce compliance requirements or audit trails. And while ML engineers are already familiar with bias and variance for model performance, continuous monitoring of bias and variance as part of the security process can help protect models and detect hidden attacks.
