The Most Overlooked Aspect of Effective Software Development
I’ll admit, this title is hyperbolic and “clickbaity”, but hear me out. If the title plainly revealed this article is really about bug reports, you may be inclined to skip past it, and as a result unhelpful bug reports would ironically and tragically be allowed to continue their reign as the silent killer of effective software development.
Abstracting this one level, this is an article about communication and trust. At its essence, a software bug is a miscommunication. Whether it be a missed communication about an expected data structure between a web or mobile application and a server, or a missed communication about a requirement between a product manager and a developer, these miscommunications are an inevitable part of software development. However, all too often the initial miscommunication that caused the bug quickly compounds into a death spiral of further miscommunication across teams, resulting in wasted time, confusion, mistrust, and even damaged relationships. When it comes to reporting and communicating about software bugs, let’s be better.
How familiar does this sound?
You are copied on a mind-bludgeoningly long email thread about a software bug. The kind where any earnest effort to resolve the actual issue at hand is long gone, and has devolved into passive aggressive finger-pointing and conspiracy thinking. You are just trying to ship good software, and somehow it feels like you’ve landed in the “comments section” of an opinion article about the geometrical shape of planet earth.
We’ve all been on these emails before, often resembling something like this:
At their worst, like in the example above, bug reports create distrust and resentment between product development teams. At their best, bug reports can build up trust and demonstrate an earnest commitment to working together and raising the floor for your product.
Another side benefit of good bug reports is that occasionally, when the conditions are right, they can actually be useful for helping engineers resolve real product defects!
What does a good bug report look like?
Physicians adhere to the latin phrase Primum non nocere, which translates to “First, do no harm”. When it comes to building great software, this a worthy principle as well. The only — and I mean only — legitimate motivation for reporting a software bug isn’t to lay blame or score political points, it is to actually help fix the problem and make the software better. If you aren’t earnestly interested in making the software better, don’t bother writing a half-baked bug report. Remember, a software bug is a miscommunication, so don’t make the problem worse with even more sloppy communication!
Once you have the right mindset, the mechanics of a useful bug report become self evident.
State the device, OS version, software version, and any other pertinent conditions.
In addition to being extremely helpful in isolating the problem, these supporting details bring credibility to your bug report.
Just as importantly, these details can be useful for helping a Product Manager determine the surface area for the problem and ultimately the priority of a fix. We all can wax nostalgic about how sweet Android Lollipop was when it launched in 2014, but do we really need to keep paying the price for its quirks when it breaks our app now in 2021?
Device: Google Pixel 5
OS version: Android 11
App version: 4.31.0
Build: 20210130Provide solid steps to reproduce, don’t make an engineer do it.
There is nothing worse than a bug report describing an obscure issue with no known steps to reproduce it. When you find a bug, taking just a few minutes to retrace your steps can save hours of investigative time for an engineer. More importantly, this is a perfect opportunity to demonstrate some goodwill and honest effort to your teammates.
Steps to reproduce:
1. Fresh install the app
2. Navigate to alerts > configure new alert
3. Configure an alert with any day/time combo
4. Save the alert configuration
5. Return to the alert config screen and deselect all days
6. Attempt to save your changesClearly state the difference between actual and expected.
If you have ever worked on building a digital product, you know very well that the “correct” behavior of the software isn’t always objectively obvious. If you are requesting the behavior of that software to be fixed or changed, you ought to at least be able to describe what the expected behavior is, no matter how obvious it may seem to you.
Expected: The app should guard against a user saving an invalid alert configuration (no days selected). When the alert config is an invalid state (no days selected), the “save” affordance should be disabled.Actual: The app doesn’t guard against invalid alert configurations, and users can attempt saving alerts with no days selected. When attempting to save an incomplete or invalid alert config, user sees a 400 error.
Bring evidence.
Now comes the fun part. Screenshots are useful, screen videos are even better. When you are testing, always have a screen recorder accessible within a few taps. You never know when the opportunity to catch badly misbehaving software will present itself.
Also, for bonus points, capture the invisible details unseen by the user in addition to the visible evidence from a screen capture. If you are testing a browser app, enable the developer console and know where to look for error messages. If you are testing a mobile app, there are various debugging tools that will allow you to capture the invisible details such as stack traces, http status codes, and more. At Livefront, we built a mobile debugging tool called “Loupe” to do just this. We’ll talk more about tooling later.
Notes: I looked in the debugger and pulled the API requests for both the successful and failed scenarios. Here is the request body from the initial, successful request:
{
"alertPreferences": {
"configuration": {
"daysOfWeek": [
"Sunday",
"Saturday"
],
"timeOfDay": "10:00"
},
"cadenceType": "Custom"
},
"sendAlertNotifications": true
}And here is the request body from the failed attempt that returned a 400 error:
{
"alertPreferences": {
"configuration": {
"daysOfWeek": [
],
"timeOfDay": "10:00"
},
"cadenceType": "Custom"
},
"sendAlertNotifications": true
}
Consider the real-world user impact.
When reporting a software bug, exercise some judgement and pragmatism. Consider how real users will experience and interact with your product in the wild. Now, think to yourself, does your newly discovered bug seem likely to hinder their experience?
This isn’t to say you should look past edge cases or low frequency issues. Even if it is a scarcely reproducible bug, it is deserving of a bug report and place in your backlog. But having consideration for how severely it impacts actual people is not only prudent, but essential to our central goal of making the software better. Not all bug fixes will improve the software equally, and we don’t have time to fix them all, so discrimination is needed.
Impact: Although it may be unintuitive why an actual user would try to save an alert config with no days selected, we should assume it will happen, leading to user confusion.They are not merely QA tools, they are communication tools.
If you want to rise above the fray and avoid bad bug reporting, you’ll need to put good QA tools in the hands of your team. If you believe that good communication is the best remedy to resolve miscommunication, and you acknowledge that software bugs are inherently difficult to communicate, then it becomes clear that teams need a little extra help in order to effectively report and resolve bad software behavior.
To put this more concretely, imagine a scenario where an entire screen in your app is broken and you have no idea why. Each time you load that screen, you get a cryptic and entirely unhelpful error message. Beneath the surface and unbeknownst to you, there is one measly little data attribute being passed into your app that is an unexpected data type. The app zigged when the server zagged, and now you are dealing with the floating point number that broke the dashboard’s back. But, unless you have a way of actually seeing the request and response objects between your app and the server, you (and your engineers) may waste endless hours debugging an issue like this.
The scenario described above is precisely the battleground between good communication and bad communication when it comes to bug reporting. Without proper debugging tools, an issue like this is just begging to fall into the miscommunication death spiral we all know and loathe. With good tools that can actually make the invisible parts of your software visible, you can empower everyone on your team (read: non-engineers) to accurately diagnose issues like this in minutes.
A home grown QA tool.
At Livefront we so strongly believe in the power of good QA, and good communication, we built our own debugging tool. It allows our team and our clients to access a “hidden” interface underneath the app’s actual UI which reveals network requests & responses, analytics events, and feature flag settings for controlling test conditions. I like to think of it as a jetpack for mobile app QA.
Whether realized by way of testing effectiveness, debugging efficiency, or simple stakeholder education about how the technical aspects of the product actually work, the benefits a quality debugging tool are invaluable. Product managers, testers, and other stakeholders love it because it empowers them to see the invisible aspects of the product, which are just as important to function and performance as the visible aspects. Engineers love it because it helps anyone, literally anyone, write a quality bug report that boosts signal and reduces noise to ultimately help discover the cause of the problem.
If you care about building great software, and you realize this process is messy and complicated, then put good testing tools in your hands and take a little extra care and thoughtfulness to make your next bug report just a bit less messy. Your team, and eventually your users, will thank you.
Matt loves pushing bug reports with lots of invisible details at Livefront.
