Coding the Big Picture

Published in

Fuzz

9 min readApr 24, 2018

I recently wrote an article (“I suck at QA. You know who doesn’t?”) in which I explain the impact an automated QA effort could have on your organization by freeing up your critical “breakers” to do what they are good at. I also recently read an article titled “How lines of code kill humans.” which describes — in simplified terms– how developers don’t always think about how their code interacts with the world. I wholeheartedly agree with this article, and I think it did a better job than I did in discussing some topics, and so I decided to follow up on my previous article.

QA is beyond a person on your team or finding bugs; it’s about finding weakness in your product and making it stronger

Due to the hyper-pace of the software development world, there is a tendency for everyone to get into their lane and now we are on auto-pilot trying to finish the backlog in time for release X. We expect everyone to be mindful of each other while we (in our own lane) pay no attention to others.

The worst part of this communication failure (and that’s what it is) is that due to the nature of software development, often times the team that gets the full burden is QA. Every lane gets affected, but it’s like compounding interest on your loans or saving accounts. The further down the line you are from the change, the greater the impact of flawed communication.

Looking at the whole picture instead of our isolated viewpoints is critical, even though we often scrape by with the status quo. This wider reflection is a hard task, no argument there, but it’s a task that can produce better ROI on your product in the long term. My original article mainly discussed one facet of this: automation should free your team to think about the bigger picture, and if you’re constantly in the weeds trying to knock out the latest “critical” bug, you’re not looking at the bigger picture.

Next I’ll describe four of the typical outcomes of missing the bigger picture. Each one of these can occur in isolation, but they tend to compound one other because we ignore how a seemingly isolated change impacts the greater ecosystem.

Outcome 1: Expanding Scope

Ever start a sprint with a Story that was 4 points, then exit said sprint with a Story that is now 8 points? Same story, maybe even same subset of tasks, only now we have a deeper understanding of how it’s supposed to work, both in isolation and in the context of the wider system. This isn’t a bad thing; it’s a great thing, really, if you plan for it.

When this outcome occurs, even though it can serve as a valuable moment of education, it can lead to a great deal of frustration. Product and Design can craft a fantastic user experience and get the client on-board, only to learn mid-sprint that an overlooked nuance of functionality or an intricacy in data relationships means that delivery of a crucial feature isn’t possible.

The solution seems straightforward, no? Just do some upfront R&D and include engineering support in initial conversations to ensure everything that is being discussed is realistic and feasible. But is this task in the current sprint? Is it pointed? Is the engineer idle? Is this all the engineer does? If the answer is “no” to most of these questions, chances are some detail will be missed.

It’s a planning and communications issue; If the greater team doesn’t give this planning the attention it deserves, , the individuals on the team gravitate towards the work that is deemed immediately important, which is often completing with the task/stories in the current sprint.

So, how do we fix it? As an Engineer, I should have a rough mental outline of how I intend to implement a given feature, and ideally this should be documented for when work actually begins. This is called the design phase for a reason, and just like Designers have their toolset, we should have our own: Sequence diagrams, Class diagrams, Dependency diagrams, and so forth.

Now, not every feature requires the same level of diligence, and that’s okay. At the very least, we should have a picture of dependencies and interactions visually laid out as early as possible, and usually that’s enough to spark the brain into realizing what we don’t know. From there, it’s a matter of answering these unknowns to get a better picture.

For Product and Design, the important takeaway is to consider the impact of engineering design into the process. The engineering team will be addressing questions such as: “Is this a complicated feature?” “Do we need to understand how the communication layers of different systems interact (Sequence diagrams)?” “Are there open ended questions we need to address prior to finalizing the ‘designs’ for this feature?” Once you know the answer to these questions, you can estimate the task.

Note that this is just a simple example of this type of thinking; in practice this process is an organic beast, and as long as you account for it in your process, it can take many different shapes.

Outcome 2: Poor Implementation

Redrawn by Glen Lipka, Original by Thom Holwerda

This is likely the easiest outcome to discuss (and resolve), yet the outcome that results the most often. The tools I wrote about in my original article (Automation, Unit Testing Code Reviews, and a good QA process) become your primary means of dealing with this problem set.

This problem set consists primarily of known/expected qualitative analysis (Also known as “the things you know you should know”), which usually results in a happy path. If these basics are failing, or constantly breaking, you can’t take a step back and look at the bigger picture because you’re always fighting fires.

So, unless you’ve solved the question “given a program and its input, will it run forever?”, just add test to combat the outcome of Poor Implementation. The more test you have the more you can grow your product with confidence.

This outcome is heavily influenced by culture and team dynamics. The value your core leadership places on a given process or outcome will always be felt by the greater team, and they will gravitate towards that which is valued. If you want to set a standard of excellence you must put the most value into the processes and practices that produce excellence. Simple.

Outcome 3: Consensus on the product

As a client services company, often this is the outcome we have the least control over, but this is a problem that is also widespread in product companies. In this outcome, everyone did everything 100% correct; we have requirements, we have designs, we have tests, everything is passing and everything was implemented correctly. But, suddenly we get in front of our stakeholders, and it’s back to the drawing board. This wasn’t the product they expected at all.

My advice here is something I say on a regular basis: Stakeholders don’t necessarily know what they are asking for. They are asking for A because they have problem X. Is A a solution to problem X? Of course it is! 100%! But, when someone asks for A, do they even know what options B or C are, or that they even exist in the first place? Unlikely.

When stakeholders are prescribing solutions, a deeper dive into the problem should be done to truly understand why an individual or organization is asking for something. Once you have a better handle on the problem, the next piece is to ensure that stakeholders are involved in the ideation, design, and making processes to a degree which they won’t be surprised by the end result. This could take many forms, but it often involves some form of prototyping the end product prior to the full buildout.

Even then, you might still run into the issue of misalignment; and when you do, don’t get discouraged. Figure out out what went wrong and try to fix the process gaps to ensure that your stakeholder is more involved in future iterations. Most importantly, don’t cut corners, and adjust your process to maintain high levels of quality, especially when assessing all the different potential product risks.

Outcome 4: Incomplete Product Requirements, Incomplete Product

Even in the best scenarios, we sometimes don’t have sufficient product requirements. A very common example of a gap in requirements is error handling. Typical product requirements cover everything that is is expected to happen as the user interacts with an app, but they often don’t cover what’s not supposed to happen. This can result in products that do their job under normal circumstances, but the minute the internet is choppy, suddenly all hell breaks loose.

There is large contingent of issues that fall into this category (like the above) which can and should be caught in the design phase of a project, but often are overlooked and caught in the qa phase. To deal with this contingent of issues, simply ask what could go wrong at every step of the process.

Security requires its own deep dive, but at the highest level, if you treat security concerns as another set of “What if?” questions to be asked (along the lines of “what could go wrong?” asked during the design phase), You’ll likely do better than the average organization in incrementally improving your security moving forward.

Similar to the above examples, another discipline that addresses the same concept is unit testing. Unit tests server as a means of establishing “what are the things this piece of code does?” and “what are the things this piece of code doesn’t do?”

If you have answered both of these questions, ideally what’s left is truly unknown behaviours.

“The problem is that software engineers don’t understand the problem they’re trying to solve, and don’t care to,” says Leveson, the MIT software-safety expert. The reason is that they’re too wrapped up in getting their code to work. (The Atlantic)

Which is 100% accurate, everyone is very concerned with getting the work done and not about how the work gets done.

“Software engineers like to provide all kinds of tools and stuff for coding errors,” she says, referring to IDEs. “The serious problems that have happened with software have to do with requirements, not coding errors.”

That is, until everyone is asking how this problem got out into the wild, as we frantically try to fix it. Sure, maybe even if we did everything right we still wouldn’t have caught this issue, but it’s 100% guaranteed we won’t catch it if we never have time to look for it due to just trying to get the software out.

This was the concept I wanted to make clearer from my last article: your “QA” people are usually good at thinking about how software breaks and that’s what they should be doing. Not verifying that it works correctly when everything is correct. It’s why I shy away from saying QA is a position, it’s process everyone is involved in pushing along.

Also not saying that it’s only their duty to think about how to break it but if your team is doing things right you’ll likely have more stability to continually inspect your product and process to determine how to improve it further.