InfoSec 201: Dealing with Test Environments
[This is the first article in our InfoSec 201 Series. The series regularly provides Practical Tips for Security Professionals™ at Signal Sciences Labs. To stay in touch with the series, follow Signal Sciences Labs or visit us at signalsciences.com]
So it begins
Every morning, the first thing I do is welcome my wife to the joy that a new day brings and then sigh. I sigh because I know when I open up Twitter, I will be inundated with #InfoSec stories and #DevOps tales-from-the-trenches that put the face-palm into full effect. Unfortunately, sometimes the people responsible for driving a company in a positive direction around these topics are overworked, out of the loop or simply moving too fast.
Since we at Signal Sciences like to give back to the community, I wanted to take a stab at filling both gaps: Share some knowledge and provide solutions based on what we can learn from other’s mistakes.
This is our first article in our InfoSec 201 series and is focused on securing test environments, sometimes referred to as beta or staging.
Recent Events In The News
A few days ago the media posted about a mid sized retailer who started to receive notifications from customers about possible phishing attempts. When they were made aware of the attempts, they did their due diligence and investigated with no signs of an intrusion. A short time later, they were approached by a security firm with specific datasets related to the breach, which the retailer used to isolate the source of the compromise. Their conclusion: The data was compromised from a test site.
In much larger example, just a few months ago a major online web property fumbled rate-limiting around one of their beta sites. This party foul landed a researcher a peace time bounty of $15,000 after it enabled them to brute force security codes which in turn allowed the researcher to reset account passwords. Once again, the attack was attributed to a beta version of the web application.
These are two companies on opposite sides of the size range, opposite sides of the tech spectrum, and what I can only imagine to be unmatched levels of security resources, yet they both stumbled upon the same pain points. Insecure test sites.
What We Learned
Taking a closer look at the above, there were 2 main points which really resonated:
- Both test sites were backed by production data
- Neither site had adequate security visibility or defenses in place
What You Can Do
Using production data in a public test environment is like swimming in a shark tank with a wetsuit made of meat. Just because you can, doesn’t mean you should. Any vulnerability in your test app automatically exposes your paying customers. To make this safer, lets take the sharks out of the tank.
- Don’t use your production database, but rather an isolated test database filled with a sanitized subset of your production data.
- Replace customer email addresses with random prefix/suffixes that prevent them from being used to phish the customer in the event of a breach
- Replace passwords with extremely long randomized strings if you cannot remove them completely.
- Replace their username, first name, last name, mailing address using wordlists and random string generators
- Replace credit card details with sample credit card numbers from systems like PayPal or Stripe
- Cycle your test account usernames and passwords when off-boarding QA engineers and developers (sometimes test accounts have legitimate credit cards associated for testing purposes)
Aside from production database access, one question you should ask yourself is, “does this test site need to be exposed to the public internet?”
If you’re running a public bug bounty, then the answer may be Yes.
If you have specific vendors working on integrating your platform, the answer may again be Yes.
If there are no specific circumstances which require public access to your platform, you should take the necessary steps to either reduce your exposed footprint or ensure your security defenses are implemented. This includes adding parity between to your test and production environment security solutions such as rate-limiting, threshold based alerts and log collections and retention.
Reduce your exposed footprint and ensure your security defenses are implemented.
Another thing to take into consideration is 3rd party access to your test environment. When you have a vendor or contractor working on your application, many times they need access to their VPN, data or the site source code itself. This can open a few holes not just in your firewall, but also in your standard practices. Take special measures to reduce the scope of access to your contractors and vendors by not allowing them access to production databases, setting up time restrictions (don’t allow them to connect outside of office hours), and prevent them from checking out source code from your SCM. Just because you signed a business agreement with an entity doesn’t mean you’re immune from breaches that they may proxy.
It’s possible to promote agility around your application and business by exposing your staging environments to the internet. Before you do, make sure you’ve calculated the risks of doing so and that you have done the due diligence to protect your customers. Remember, testing your application is important to your business, securing your testing is important to your integrity.
Thanks for reading this article. If you enjoyed it please let us know by clicking that little heart below.
At Signal Sciences we are building the industry’s first Next Generation Web Application Firewall (NGWAF). Our NGWAF was built in response to our own frustrations of trying to use legacy WAFs while enabling business initiatives like DevOps, cloud adoption and continuous delivery. The Signal Sciences NGWAF works seamlessly across cloud, physical, and containerized infrastructure, providing security without breaking production traffic.