It looks like these guys didn’t do enough testing

Shifting Security Left

During the creation of an upcoming presentation, I remembered a story of the XML Bomb that I created (a version of the Billion Laughs Exploit)and how I managed to bring down a server of a payment gateway. A test server for a payment gateway that is. This was back in 2008, so it probably wasn’t too difficult to do — I made sure to make the XML Entity “wtfh4x” too.

Amusing as this memory was for me it made me think that the current state of security testing by Testers is not good enough; security is critical to developing a product that will be used by the public, but it’s still not taken incredibly seriously by Software Development teams.

Sure, there is an increasing awareness by Dev Teams of safe development practices and OWASP. However, very few QA Engineers recognise that they should also be testing an application’s security. Coming from a background of a QA Engineer in the online payments space — I know that it is especially important to understand that building security into the development of a product from an early stage can improve the quality of the delivered application.

A few scary numbers

It is estimated that 2.5 quintillion bytes of data are being created per day. That’s 2 followed by 5 followed by 15 zeros. In February 2018 legislation was put into place to control data protection, data collection and data usage of the ever increasing amount of Personal Information Data. Along with this have come data breaches and wider spread of security concerns. Threats and data breaches are ubiquitous, ongoing, and sometimes state-sponsored. In the healthcare sector in Australia in the 6 months between February and July 2018 there were more than 300 data breaches in Australia, showing just how vulnerable our data is.
Therefore a good level of application security is relevant to every organisation that is dealing with customer data. In the online payments industry, GDPR and PCI DSS compliance is something that needs to be thought of during the inception of a product rather than bolted on as an afterthought of “oh yeah, we should do some security testing” and something that these online payments companies have had a lot of experience with, but a high level of data security is no longer just for financial organisations anymore.

Test early and often

A report called “State of DevOps 2018: Strategies for a New Economy” was released in 2018. This gloriously data-heavy report highlights many issues that define organisations; from the Ultra High to the Low Performers. One really interesting area that the report advises on is that integrating security earlier in the software development process is one of the key technical practices that drives high performing teams.
This introduces the idea of shifting security testing left — which in other words is saying test early and often. Shifting left is an approach in which a task is performed earlier in development lifecycle (i.e., moved left on a project’s timeline). The ROI of shifting security left can be worth it. It saves time, money, company resources and if a security flaw is found once a product is created and is in the market — the cost of remediation for this issue can be huge.

Fundamental principles

What everyone involved in Software Development should know already is that these ideas of streamlining and efficiency of software delivery make perfect sense. The idea of automating all the things is fundamental to being an Ultra High Performing organisation. Releases are getting easier to do these days and there’s very little reason to not take advantage of automating as many of your organisations painful, repeated processes as possible.

Automating as many of the processes as possible means that it shouldn’t take days or weeks to get new developers up to speed to set up and deploy code for a project. Of course, they should learn about what the application does that they’re working on, but to get a local copy of the code up and running and any fixes committed and deployed should not be an arduous process. There should also be automated tests in place that can be run that increase the confidence a new Software Engineer has in the new code base. Architecture is difficult — and expensive — to change. It’s easier to work on processes and make the path of creating and releasing software as painless as possible.

A successful software project needs a collection of sensible and comprehensive requirements. These are split into two basic areas; Functional and Non-Functional. The distinction is important, especially for QA, as we need to know the boundaries of how a system should and shouldn’t behave. The Functional Requirements are basically things that have a function that can be measured and quality that has already been defined, like a “button press will do this” or “this page will display these things”.
The Non-Functional requirements are traditionally behaviours like Performance, Load, Stress and Security. They can be termed as “ility Tests” — things like Availability, Reliability, Serviceability, Maintainability.

Testing of these NFRs are usually pushed to the latter stages of a project when it’s pretty difficult, and pretty costly, to change basic Non-Functional Requirements I’ve found that a few of them to get tested during development of software, but Security Testing still lags behind.

It’s an old joke, but not really a joke. It’s the sort of thing that QA Engineers should be testing. Walking into bars and ordering beer is optional though….

A QA walks into a bar…

It’s an old joke, but not really a joke — it’s the sort of thing that QA Engineers need to be bearing in mind when testing or creating test cases. There are many ways that I can think

A favourite way of mine to find some bugs is to consult the OWASP Top 10 and test a few examples from that — add some XSS JavaScript to a field or drop in some iframe tags, or try to upload a Billion Laughs XML. Tests such as URL fuzzing (basically just adding variables to a URL to find hidden pages or error messages) could take place during the SIT process, to check how the app behaves with unexpected URLs. This could also include a lot more testing of invalid URLs or the “sad path” flows through apps, as these cases should be correctly handled by the app.

Tasks such as attempting to break character limits of fields would also be a good use of automated testing during app development. There is an amazing free plug-in for Chrome called Bug Magnet. It allows you to test any field on the web page with a huge range of different format exploits, language characters, special characters, large strings and lots more, all at the click of a contextual menu on screen. The range of free tools available to test a website is immense, from browser plugins that instantly clear the cache, to tools that will modify the headers of the request that you send.

As QA Engineers it is important to remember that there is a wealth of free tools available that can give inspiration on how to break the functionality of an app and most browsers have this. It is called the “devtools”. It helps to become familiar with this concept — it can help with everything from navigating the DOM or setting breakpoints in the rendering of Javascript or throttling the browser’s network connection. And all of these can help to reveal bugs and potential security issues.

Below is a list of the ideas for manual high-Level security testing, however, all of the concepts of these tasks can easily be automated too:

  1. Use examples from OWASP Top 10 such as XSS or XXE,
  2. URL fuzzing,
  3. “Sad path” testing (use the app how it is not supposed to be used),
  4. Attempt to break character limits,
  5. Use a tool such as Bug Magnet,
  6. Modify the request headers,
  7. Spoof the User Agent,
  8. The browsers’ devtools are your friends

Going forward

Anyone involved in the development of an application should have an awareness of the security of what they are creating. They should have awareness of not just safe coding practices, but with more security conscious manual and automated tests that happen early on in a software product’s development.

These can be included in CI/CD builds, where basic security tests are run on the code at the time of a deployment or a code check-in. High Performing Organisations are already working in a manner like this, where committing working, tested, highly secure code to production every few seconds is standard practice. Other secure development practices could include incremental scanning of code, code linting to find, and fix, common coding mistakes and highlight insecure code.