Helping Companies Better Protect Themselves By Performing A Penetration Test Instead Of A Vulnerability Test

Hellfire Security
4 min readMar 31, 2023

--

To demonstrate what a difference a penetration test makes, we will compare a vulnerability test against a penetration test. Once we have, you will understand the benefits one gets from running a penetration test instead of a vulnerability test. To make it fair, we will compare a vulnerability test and a penetration test that happened to the same company, a small fintech startup located in Thailand. Both tests were internal tests and happened within one year of each other. Now, let’s compare the two. Let’s see which test got the better results.

Identifying A Vulnerability Test

The vulnerability test was performed by three testers. The penetration test was performed by two testers. The vulnerability test team got a lot of help as they received the network architecture, subnets in-use, and list of enterprise applications. The penetration test team didn’t get any help. In fact, the staff did not even know a test was happening. As you can already see, the vulnerability test is starting to look more like an audit while the penetration test is looking more like what you would want, which is hacking. Employees fight an audit. Employees hide problems from an audit. Hacking operates in stealth and avoids any such interference.

Implications Of Vulnerability Test

Being seen by the staff, especially the security team, there is an opportunity for the staff to protect themselves. Any time that happens, vulnerabilities are missed because there is a deliberate effort to hide them. I didn’t say fix them. I said hide them. This is exactly what happened during the vulnerability test. The staff hid as many vulnerabilities as they could from the vulnerability testing team.

There was no such problem during the penetration test. Operating in stealth, nothing was hidden from the penetration testing team because no one knew that they were there. More importantly, having a greater degree of skill than the vulnerability testing team, they didn’t set off any alarms so they stayed unknown to the staff the entire time that the test was going on. Why would we assume that the penetration testing team had a greater degree of skill? They didn’t get help. They had to find their targets on their own. They had to be quiet so as to avoid setting off alerts. We knew that they were more skilled because they had to be to get anywhere on their own. Not being obstructed, having a greater degree of skill, the penetration testing team was bound to find more than the vulnerability testing team, and they did.

What Was Missed By The Vulnerability Test

Having it easy as they did (getting all that help), the vulnerability testing team would not have taken the time to understand their target. They would not have reversed engineere the environment to find the most likely weak points. Instead, they would have just gone through an exhaustive list of known vulnerabilities. Each would have been launched at the target without any concern as to whether or not the attack would be successful. What that meant was that the vulnerability testing team wasted a lot of time on attacks with little or no chance of success. They wasted a lot of time on attacks that even if they were successful would get them little to nothing. It’s true that they compromised Active Directory. However, the target didn’t use Active Directory for anything important. Therefore, all they ended up being able to do at the end of the test was listen to staff voicemail.

What Was Found By The Penetration Test

Unlike the vulnerability testing team, the penetration testing team reverse engineered the environment. They made sure to understand the architecture, the network, the people, and their processes. Once they understood that, they let the people lead them to their targets. Once they saw those targets, they knew what would most likely work against them. The penetration testing team spent their time on those attacks and didn’t waste any time on anything that wasn’t going to pay off. More importantly, they took the steps necessary to position themselves so that the attacks had the highest chance of success. They got an initial foothold, elevated their privileges, moved laterally, and finally waited for the right time to launch their attack. All of this was done to increase the likelihood of their success. In the end, it paid off. They were able to compromise the network. Crossing over the network was an unencrypted backup of the database. The database contained customer PII, customer credentials, and the customer’s OTP seeds.

What This Means For You

The vulnerability test was carried out by one of Thailand’s premier testing teams. That team is very expensive. For all the money that the customer paid, they didn’t get much. The penetration test was carried out by Hellfire Security. With Hellfire Security, the customer got much, much more. That makes Hellfire Security a much better value. More importantly, it means that the customer does not have to waste time and money on vulnerabilities that don’t really matter. They can instead focus on those small number of vulnerabilities that really do, and that means saving them, the customer, both time and money.

OK. That is it for this blog post. In the next blog post, we will finish up our superficial look at Thailand’s perimeter. The first time, we used Google dorks for our superficial look. This time, we will use Shodan. Hopefully, we’ll see you there.

--

--