Recently, I was put in a situation that many internal red team and penetration testers face. A product needed to go live by a certain date and as usual security was brought in to certify that it could be put publicly facing at the eleventh hour. This left my team and I with one week to preform a thorough security assessment. The product team expected us to roll over and just preform a half-assed vulnerability scan. I have never sacrificed the quality of the work I do for the sake of deadlines and wasn’t about to start now. This left me in a tight spot. I had one week before the product would go live, and a front, a CMS, and an API that needed to be tested. Not just tested, mind you, but any issues needed be addressed and remediation verified prior to product launch. It was a seemingly insurmountable task. However, I am an optimist at heart, or maybe just a crimudgen that refuses to accept defeat, and when you have a gun to your head, you improvise, you adapt, and you absolutely do not fold like a lawn chair. No matter how much you secretly want to crawl under your desk and wait for it all to be over. The result was one of the most frustrating yet efficient penetration tests I have ever preformed.Now I’m not a big fan of buzz words. That said, I’m about to use a couple of them. What came out of the problem I faced can best be described as an Agile Web Application Penetration Test. What I’ve come to realize after talking with a friend who is now working in the newly termed niche of SecDevOps (buzz word #2) is that this is what he’s working toward.
Let me explain the process that I slapped together in literally half a day. Each morning we would start with a level set call. The call would include the developers, product managers, myself, and another member of my team that was also assigned to test the application.
We would start the call by reviewing any new findings from the previous day. We would explain what we found, and the developers would confirm there was an issue or not and we would hash it out directly on the call. This saved us a lot of time in collecting and documenting in excruciating detail every flaw found. This also helped with any questions the devs might have had on the flaw and we’d discuss as a group how best to address them. This was how the developers would get their influx of work.
Second, we would discuss any fixes that had been put in to place by the developers on the previous day’s code push. We would then validate the fix was adequate or work through how the fix had not been successful. This discussion helped expedite the remediation and validation process from days or weeks to hours. Having several eyes on an issue from both sides of the fence along with an open dialog about how to fix it, produced some impressive results for solutions at breakneck speed.
The last part of the daily call would focus on the penetration testing activities for that day. We would go over the aspects of the application we would test for that day and make sure we weren’t bumping heads on any areas that would be updated that day. While code push usually happened at night, we wanted to make sure that the we weren’t going to have the rug pulled out from underneath us from a major code change mid-day. We would address any concerns with the areas of testing and then break from the call and go about our testing. The call would last at most an hour.
With the call completed, we would go about our testing for the day, focusing on the areas that we had discussed and agreed upon during the call, collect findings and then report back the next day.
The time from issue reported to issue remediated (for this test at least) was usually a day or two. The other tester and I also weren’t required to document down to the most minute detail how the vulnerability was exploited or how the vulnerability was actually a vulnerability. This also saved a lot of time with the final report as we weren’t required to prove the vulnerabilities to the dev team. We demonstrated and confirmed flaws and misconfigurations each morning on the level set calls. I think the final report was only about 10 pages long.
Don’t get me wrong, this process was equally frustrating. With the quick fix issue remediation that was taking place, we ran into a couple of situations where we would find a vulnerability on one day that would be fixed the next when we found another flaw that if chained with the now fixed issue, would have allowed for more serious exploitation or information disclosure. This is only bad in that we (the offense) felt like we hadn’t necessarily won. I understand that this is only a minor bruise to the ego not being able to pop a shell, or dump a database, or reset an admin password from a lower privileged account. So not really a bad per se, but still a bummer for us testers. We as attackers always want that shell. It’s something that’s ingrained in our psyche. If we didn’t get that almighty shell, we have somehow dishonored our Red Team ancestors. However, as my friend so eloquently put it, “I think the larger point is that we need to evolve past popping shells and getting root but helping identify and fix low level code issues that could result in that outcome. If DA is still your goal, you’re not working as a team”.
After a week, we had found wide variety of issues with the application that were addressed by go-live. The issues ranged from minor things like session timeout and security headers to privilege escalation and other, more serious issues. All of these were fixed in time for the product to go live. I was seriously impressed with everyone’s ability to work together in this completely foreign process that had been slapped together in haste and out of necessity. The development team was awesome and really impressed me with their eagerness to fix the issues. Their team worked well with us and us with them. There was a mutual respect and understanding of working towards a common goal that I think was key in making this work. In addition to everything I learned on this wild ride of a pentest, I also learned that sometimes that shell isn’t what you should be hunting. Rather, you should be looking at the application and test as a whole, and if the little issues you found and fixed along the way build a stronger base application which prevents you from achieving RCE or other such big bad, then you have accomplished your goal. It’s not about making a thick report and laughing at the developers for some perceived incompetence. It’s about working as a team toward a more resilient and secure product. Something that many on the red side of the house (myself included) often lose sight of.