Application Security Best Practices

We recently hosted a webinar on application best practices. CryptoMove CEO Michael Burshteyn and Senior Security Engineer Curtis Davis discussed best practices for application security, discovering risks in your environment, and the advantages of moving target defense.. You can watch the recording here:

Read on for a transcript of the discussion.

[0:00:00] Mike: Hi, so this is Mike. I am the CEO of CryptoMove, we’ve got Curtis here who is Senior Security Engineer. And we’re here doing a webinar on application security focusing on keys and secrets management and trying to look into the possible ways to threat model keys and secrets in an infrastructure environment and understand how to best approach it, security best practices.

The format of the webinar would be about 20 minutes from Curtis going through a deep dive and then the final ten minutes will actually go over moving target defense and explain a brief background on that which is something that CryptoMove is working on applying moving target defense to keys and secrets managements to help secure cloud-native applications. And we’ll do a little walk through of our roadmap as well. So I am going to turn it over to Curtis to walk us through application security and keys and secrets management.

Curtis:Thanks Mike. Just a little introduction, my name is Curtis Davis, I am a Senior Security Engineer at CryptoMove. I have been here over a year and a half, doing everything from penetration testing to application security testing, STLC code review. I code in python as well for automating security tasks, some DevOps stuff here and there. But today we’re going to be talking about keys and secrets: basically how to protect them, how to secure them, how to discover them in your own environment and some tools to help you guys do that.

We’re just get started here with the first slide, which is secrets and keys. Basically what are they? An API key or secret is basically code passed in by a program calling an API interface to identify the calling program, it’s developer or its user to the website. API keys often act as both a unique identifier and a secret token for authentication [0:02:00] and generally have a set of access rates to the API associated with it. Nowadays, some of the complex APIs will have granular access rights for these API keys that tell it what it can and can’t do. There are some APIs out there that don’t enforce granular API access rights and just basically once you’re authenticated you have open access to everything inside that interface. So that’s kind of a dangerous situation, where without access rights to your API keys, if anybody gets a hold of them it can be pretty dangerous to your organization. To add on to that, it’s up to the developer of the API to build in these particular access rights, otherwise the API keys generally have wide open access to the data inside of it.

Secrets and keys are points of weakness in an otherwise secure system. So even if your system is totally secure, if one your API keys gets out into the public, it’s dangerous. Anybody that steals that API key is impersonating you or your program to that particular interface and it becomes very, very dangerous even if your system is totally secure. Again, it’s available to any and all repository contributors, clone copy distributed, anybody that has access to public repos like GitHub or private repositories as well. And again each copy of that secrets authorizes access to anyone that gets hold of it.

So, let’s go over what a secret is, what it looks like. There are a ton of kinds of secrets out there. We have API keys, anything, it’s just a series of letters and numbers, you’ll see to the right of that, that goes inside your code and authenticates your program to that application interface. AWS keys are a point of contention for some of the recent breaches in 2017 and are a series of letters and numbers that are usually longer than generic API keys. AWS keys tend to be 20, 30 characters. [0:04:00] And then we have oAuth client secrets. Again it’s an access token, you’ll see to the right of that it has an expiration date, some of these don’t. Some of them are essentially good until they are regenerated by the developer themselves.

And then we have SSH private keys. So again this is authentication to your severs and these SSH private keys have to be stored somewhere securely, otherwise if somebody gets a hold of it they can SSH to your server without any credentials at all because this is the key to getting onto your server.

So, let’s talk a little about some recent data breaches, there were three in particular in 2017. The first one I would talk about is Uber. Basically in the Uber data breach, the hackers gained access to Uber’s private GitHub repository and they nixed the company’s AWS account credentials, [indiscernible 0:04:56]. Then logged into Uber’s AWS account and downloaded files that included millions of app users’ names, emails, addresses, phone numbers, drivers license numbers etc. So really bad for Uber. There was a lot press on this. So not only reputation damaged but they can get sued as well and they did.

The next one is OneLogin. An attacker obtained highly sensitive keys for AWS again from an intermediated host breaking into its service. Basically using its front door key, so some of the keys that I talked about with one side keys, it’s a front door key to your infrastructure especially in the cloud.

And then the third one is DXC also in 2017. Reportedly an IT member accidentally uploaded an outsource frame private AWS key to a public GitHub repo and in this the time they discovered it was only 4 days…in that time the attackers used those stolen keys to start 244 computers resources and power up a $64,000 AWS bill, so pretty dangerous [0:06:00] stuff.

So, let’s talk about some best practices for securing your keys and secrets. Nowadays, we know large organization can have 100’s and thousands of keys in their environment quoting around that developers use for authorization. And services are in core business component. Some of them are just in public service and plain text are on the public commit to public repositories. And the question for security folks like myself is how do you make it harder to accidentally commit secrets, so putting those safeguards in place before the code actually hits your repository and then detecting, if and when it does after the fact, right. So if something does happen the security folks are aware of it and they can then handle that situation appropriately after the fact.

So let’s talk about some open sourced, free solutions that are out there, that security teams can use to detect secrets inside committed code and then prevent secrets from being committed to those repositories.

The first one I want to talk about is truffle Hog. Basically this tools searches through bit repository for high entropy streams and secrets, digs deep into commit history, tries to find those secrets inside your code and again these are GitHub repository specifically.

The next one is Git-rob. Git-rob is basically a tool to help find potentially sensitive, push the public repository like GitHub. Git-rob looks on repositories] to use your organization and then iterate through those commit history and flag files that make signatures for potential sensitive files, so looking for specific signature for sensitive files, all keys and secrets inside your committee code.

Then after it has findings that would prevent that to a web interface and you can browse through and analyze it and do whatever you want with the data.

The next one is Git-secrets. This one is actually created by AWS labs and so, AWS created this one. Basically this tool prevents you from committing any passwords or other sensitive [0:08:00] information to GitHub repository. So it’s got some regular expressions that it uses. A few other tools that will help you as an organization or your developers from committing those secrets.

The fourth one was actually created by auth0. Repo Supervisor is the servers tool that detects secrets and passwords in your pull request one file at a time. So it iterates through each file inside your pull request and looks for those secrets before they are committed into code — another great tool for that.

And then the last one and probably the best one in this list was created by Yelp. It’s called detect-secrets, so it has a few different modules to it. The first one prevents new secrets from entering the code base and the second one is actually detecting if these preventions are bypassed, checking your code after the fact. So not just preventing it but also a server detection tool on top of it, to detect secrets that were committed already maybe in the past maybe they were overlooked before you put the tools in place or were bypassed by the prevention method and then detected afterwards. So it would detect those as well.

One thing I do want to mention is I believe we’re doing a webinar next week to do a deeper dive into some of these tools so be sure to check that out. I think Andrea would be sending out some information about that too.

Things you might want to consider. So whenever you are choosing the tool, we talked about two ways to detect: preventing before they are committed and then detecting after the commitment takes place. So something you want to look for in a tool, I just wrote some bullet points here. The first one is support a white listed baseline of secrets. There might be existing secrets in your copies or Git history but you don’t want that to block the deployment of any tool you put in place. It’s better to assert that all commits after a certain point in time are free from secrets. At a later time you can go back, you can review those secrets that were already there before the service [0:10:00] was deployed and then retroactively roll and improve their stories that way.

And the second bullet point integrates with your software development life cycle (STLC). Basically, you want to choose an optimal solution that works well with your developers workload, because secrets management is an ongoing process. So rather than enforcing this tool on them through periodic audits or forcing a tool that doesn’t work with their workflow from the get go, you want a solution that integrates well, that’s easy to use and that your developers can get behind because if it’s not easy to use your developer is not going to use it. And it’s pointless to roll out a tool like that. Something to keep track of there.

The third bullet point is compatible with the pre-commit framework you use. Some companies have only leveraged their framework to run assertions before committing to their code base. An optimal solution like this would build up on this framework and maintain backwards compatibility with the existing hook installations. So again, you want to make sure it integrates well with your environment. So don’t choose a tool that’s hard to get behind, it’s hard to integrate, especially with your pre-commit framework.

The fourth bullet point, we are talking about lightweight and modular, basically you want a tool that is easy to customize so if you don’t find the one that does exactly what you need, you want one that is easy to customize, easy to change and fits your needs within your larger ecosystem.

And then the fifth bullet point, language agnostic. You want it to work with all languages. You don’t want it to just support one client, so make sure you choose a tool that fits with your language that you guys are using or languages that you might use in the future.

So the last thought here is, you’re not [0:12:00] going to find a catch all solution. You want an ideal solution that leverages the client side preventative solution we talked about earlier and the server side detection methods to prevent the secrets from entering code and also scan your code base after the fact. Just one of these solutions isn’t enough for most environment, you want both of them to really have a heuristic view of your environment.

I think that’s sort of the conclusion of keys and secret and how to prevent getting them into your code base and also how to secure them after the fact. I think at this point I am going to turn it back over to Mike. Have Mike go over sort of what CryptoMove does, how we can help secure your secrets and the roadmap moving forward.

Mike: Thanks for the awesome overview. And everything Curtis just said will be recorded and we will roll it out and transcribe it so it would be easy to access by your teams.

So in this section I just want to briefly go over moving target defense which is what we’re working on and how that can be applied to keys and secrets management. And we plan to do these webinars pretty often, every week or two. Our thought is to keep them short and digestible so that they can add as much value without taking up too much time.

So basically, the concept of moving target defense is to take an attack surface, any type of infrastructure whether that infrastructure be at the application level, the network level or the data level or cloud or vEdge and make it a moving target, so that it continuously shifts and changes…morphs over time. The point of this is to take the number one advantage that adversaries have today which is as time goes by they can hide in the system, map [0:14:00] it out and plan their attacks. To take that advantage and actually turn it into a disadvantage so that as time goes by things actually continue to move, change, shift and time is now a disadvantage. So that’s sort of a basic idea of moving target defense.

If we look at some resources that you can check out around moving target defense on the next slide, basically we’ve compiled some of the top resources. Moving target defense has been researched at the Department of Homeland Security which is DHS. MIT has a great study cataloging various moving target defense tools. There is some really interesting threat modeling and the ability to use game theory or other risk-based approaches to actually quantify the reduction in risk based on moving target defense approaches and apply them. So these are just some great resources.

On our blog, we have actually published a compilation of them so it’s an easy one stop place to check them out.

On the next slide, we just have a quote from the Department of Homeland Security explaining a little more in detail this concept of shifting the attack surface, flipping the asymmetry. It’s all about changing the infrastructure across as many dimensions as possible. The whole point of moving target defense is that perfect security is not possible to achieve, so our job should be to make things as difficult as we can for adversaries. And even if we can’t detect them, it still makes it tough.

A few analogies to provide some mental frameworks for moving target defense that we found has been effective. One of these is the idea of nuclear weapons and how they are protected at sea. They are split up, spread out and continuously move around on submarines.

Another one here in the next slide is just this idea of asymmetry and taking high power, small groups and being able to [0:16:00] defend against a massive large group. This actually comes from a pretty hilarious set of YouTube videos that I highly recommend you check out. It has things like a thousand Jedi’s versus 25 thousand Spartans or 300 dwarfs versus 3 marines and things like that. So definitely check that out on YouTube and we’ll put a link into the chart for you.

The next analogy is pretty basic and it’s a simple one: it’s a shell game and this actually comes straight up the Department of Homeland Security website. The idea is even if you are watching it, it’s difficult to understand where the target is because it continuously moves over time.

Another way to think about this is the attack curve. So today you can think of the attack curve as the red on this diagram so as time goes by the difficulty of attack goes down. Now with data security especially and key management, you can increase the key length, increase the strength of your encryption algorithm and that would marginally increase the attack difficulty. But as time goes by it always approaches zero. So the idea of moving target defense is to literally flip the curve upside down and make it so that the entropy of that system stays constant or even potential goes up.

If we go on to the next slide, we can see the threat model around moving target defense and specifically applied to data protection. Moving target defense can make the attackers’ return on their investment lower. So they have to invest more resources to map out a constantly changing infrastructure. It can also help to protect against threats to distributed infrastructure. Where it is difficult to identify all the point of attack and detection becomes an asymmetric process that actually costs the defenders more as infrastructures gets more and more distributed.

In terms of cryptographic attack, it’s really interesting. So to attack any type of encrypted data, you need two things: you [0:18:00] need the key and you need the encrypted data. Most of the innovation around data at rest protection has been focused on how to make that encryption algorithm stronger, how to make it harder to brute force the key, how to make the key length stringer and things like that.

At CryptoMove, our own invention is that we flipped it and we said let’s just make the data itself a more difficult target. A completely different approach, a different way of thinking and as I mentioned you can quantify this, it’s now just theoretical and so we have some good resources on that.

In terms of the next slide, there is a really interesting PhD desertion that was published on moving target defense. It provides a framework here to understand the effectiveness of various moving target defense approaches based on math. And we have really interesting math around CryptoMove, around simulations, active red teaming, surveys and operational environment as well, across the vectors that you see which range from security. It’s obviously incredibly important, but also what’s the performance impact? What is the cost of implementation? This is something Curtis mentioned which is something that works in your environment and it is part of your DevOps workflow. So this is an interesting way of evaluating various types of MTB tools.

Over here on the next slide we’ve got another excerpt from that same research paper basically. It’s all about the spectrum of moving target defenses. You can think about moving target defenses across the spectrum from high effectiveness, low cost to low effectiveness, high cost. And obviously you want to have things that are really effective and don’t cost as much resources. So obviously we put CryptoMove into the best part of this graph. We’re not biased or anything but we do have some really interesting case studies with the Department of Homeland Security support [indiscernible 0:19:56] and hundreds of active data users from hotels, [0:20:00] airlines banks, all the major tech companies, startups actively using CryptoMove because it is so effective and actually improves the efficiency of DevOps environments and things like that.

So, moving on to the roadmap, and what I’ll do in a second here as we move on past this slide is actually walk briefly through CryptoMove and what it looks like. We can make the private beta available to all registrants of the webinar. We’ll make sure to send that code out. What we’re hearing in terms of feedback from our users this far is to enable the following features and we’re currently working on them, prioritizing them, and constantly iterating on the beta product.

So, one is to make the API accessible externally so that developers can actually use CryptoMove and build it into their own applications and workloads. Or making keys and secrets shareable so that they can be worked on across a team in a DevOps environment. We’re adding all sorts of different tagging functionality from metadata to understand whether it’s a dev, prod or testing environment, whether it’s an AWS Azure the type of sensitive. Many organizations have all sorts of different scales from one to five — gold, silver, platinum, secret sensitive and all sort of top secrets things like that, writing analytics. We’re adding all sorts of functionality to dynamically generate keys so that you don’t have to actually manually create or enter your own keys. You can just select different cloud based services to do that. Within a few weeks, actually ahead of schedule, I have some file attachment functionality so you can attach things like m-files or upload files for your APIs. One awesome idea we got from a user was to develop a Slack app to discover secrets and Slack gets the secrets, puts them into CryptoMove, enables them to be shareable and other types of T discoveries that we’re working on as well as [0:20:00] making CLI access and language libraries available.

So that’s kind of the upcoming roadmap on this. I will share my screen here actually to quick walk through the current beta environment so you can just see this here. Basically this is our live beta, as I mentioned 100’s of developers using it. We welcome as much feedback as possible, you can do things like enter your secrets, name them. And if you have AWS credentials and things like that, there is advanced metadata tagging where you can put in a description field and set expiration dates through your keys.

And when you upload them, basically they go into CryptoMove, they get split up and they get moved around. They are going back into the CryptoMove clustering in the cloud, it’s all hosted in the cloud and can be actually containerized or put into customers’ cloud environments and are able to basically manipulate the keys and do things like create new versions, rotate them and add descriptions. Moving forward, we’re actually about to have a new release. If Andrea’s got the slides, I’ll switch it back to her and I can show you guys what is coming up next. And so basically we would have a total revamp on the UI, API accessibility and sharing. So that’s just the next slide after this here.

So this is kind of the new release on the right. You can see the mobile version what you’re working on. On the left, you can see some of that tagging that I mentioned. You have this mouse over, the shared secrets displays how [0:24:00] the secret would be shared and things like that. And then the next part you can see things like our authenticator which will go live in the next release.

So yeah, we also plan actually to do weekly product demos as a recurring webinar to it’s really easy to check them out. And we will also on doing these informational and educational webinars as well.

Thanks everyone for listening and participating in the CryptoMove webinar and we look forward to your feedback and continuing to iterate the product based on the great and helpful feedback we receive. [0:24:38]