Living in the Grey

Black, White, and Grey Box Testing

The world of software testing is changing, and it’s not just about checking for errors anymore. I can say that the more time I spend in the testing communities, the more I hear about test strategy and quality coaching, which is something the early years of my career were sorely missing. As the pace of development has quickened, testers have also needed to be quicker on their feet, covering more ground in a shorter amount of time.

For most of my education and career, I’ve found teams trying to divide testing into smaller buckets. In practice, testing falls into a spectrum. Instead of white box or black box, I find myself spending a lot of time in the grey area between them when I’m trying to make the best use of my time.

How do we know where to focus our testing efforts?

White Box Testing

Completed from the perspective of the development team, white box testing requires knowledge of the inner workings such as the code, architecture, and business decisions. So often the responsibility of white box testing falls to developers who are struggling to keep up with the demands of each sprint. For testers, it can feel out of reach if they aren’t as proficient at coding.

By having direct access to the code, a tester can test boundaries and paths because they are easily accessible. This is best for finding security flaws, data flow errors, and things you wouldn’t normally stumble into like you might during exploratory testing.

This is my go-to method when I’m worried about parts of an application that haven’t had as much attention as others. Increasing coverage and testing algorithms directly can bring a little extra peace of mind.

Black Box Testing

On the opposite end of the testing spectrum was Black Box testing. Done from the user’s perspective, there is no understanding of what’s going on in the code, or what decisions were made from a business perspective. There is only the expectation of what should happen based on the clues presented. We apply learned cues to explore the system as a user would, hopefully questioning anything that deviates from the norm — like design choices that leave us wondering what’s supposed to happen next.

The thing I love most about new team members and new testers is the fresh set of eyes they bring to a project. Without preconceived notions about the system, spotting questionable system behavior comes naturally. I worked on one project that loaded slowly between step 2 and 3. We were all so used to it that it didn’t even register as a problem anymore but a fresh perspective brought up new questions and corrected the problem.

Grey Box Testing

I don’t hear the term much, but somewhere between white box and black box is where a lot of testing is completed, particularly in the web-based applications that are so pervasive.

Grey box testing is precisely what you would expect by the name — a little bit of white and a little bit of black.
Anyone who works on a system for long usually ends up working in the grey area. True black box testing means the internal workings of a system under test (SUT) are completely unknown, but it’s impossible to not learn something about the system eventually — even if it’s just the way that your devs think. On the flip side, it’s nearly impossible for any one person to know all the internal workings of a system, particularly if it’s older or complex.

Being able to refer to code, user stories, diagrams, or other documentation to assist in the testing is like having a cheat sheet in your pocket. Using dev tools in the browser, referencing error logs, mob testing… these things all fall in the grey area between white box and black box testing.

This has been one of the best tools for bridging the skill set of the developer part of my brain and the testing side. In fact, I’ve learned so much about testing from my coding experience, but even more about coding from my testing experience.

Embrace the Grey

Fun fact — grey has always been a symbol of balance, stability, and reliability. For me, these are things I look for in testing software so it’s easy to embrace the idea of finding a balance in testing methodologies. Figuring out which box is the best for your needs is most of the battle.

Here are my initial thoughts on what to use and when (Let me know if you think differently!).

You don’t have to know everything about testing or coding to live in the grey spaces. For me, it was about asking questions — particularly my favorite, why.

Whether you’re looking to learn more about coding, testing, or a little of both, a great place to check out is Test Automation University. Their tracks have everything for beginners new to testing through experts looking to polish skills.