Pretty in Burlap — How perfection gets in the way of shipping code
The biggest risk associated with shipping a new feature is that users don’t want to use it. But what if the reason they don’t want to use a feature is something small, like the way form fields focus, or the way a certain section animates, or a lack of tooltips, or the colour of certain buttons? What if the reason they don’t want to use it is that it isn’t “easy to use”?
This is a story about the dangers of prioritizing ease of use, how Bench ran into them face-first, and the tools we used to help us overcome them.
Trying to build it right
Picture this: You’ve decided on a new feature. It’s big, with lots of new UI, API endpoints, and operational impact. You need to make sure the rollout goes smoothly, because if users don’t have a good first impression they’ll probably never look at the feature again. That would be bad, because the executives are counting on this feature to increase engagement and reduce churn. Ideally you want to build it pretty quickly, because a bump in the numbers this coming quarter will be great for next quarter’s fund-raising.
Still with me? Welcome to Bench Engineering just over a month ago. We were working on a feature called Transaction Tagging (the ability to tag transactions based on Line of Business, Location, and Client), and we really wanted it to go well. We wanted it to go so well that we were behind on our milestones, and our commitment to a Beta release at the end of the quarter was looking less and less realistic.
This was pretty frustrating because overall we were doing a lot of things right. The Product team had done great research and prep work, so the problem we were trying to solve was well-defined. The Design team had built a clickable prototype that we gave to real clients to validate interest in the feature. Our team was working through tickets and actually deploying code to production several times per week behind a feature flag.
So why hadn’t we gotten the feature in front of our clients?
Paralysis by good intentions
At Bench, we take pride in making bookkeeping easy. Transaction Tagging is different from most of Bench’s technology in that it is the first feature we’ve built that allows our clients to do data entry. Frankly, this made us nervous. We were worried that if clients didn’t think this feature was easy to use it could have a negative effect on their perception of Bench’s ability to make their lives easier.
“Easy to use” is one of those statements that eats deadlines for breakfast. When it starts getting thrown around, the definition of done for a feature changes from “are they able to do it” to “are they able to do it easily”. When this happens, we start to see the goalposts moved backwards with every ticket, because anything — animations, keyboard shortcuts, error handlers, autofocus, text wrapping, hover states, components designed for future complexity, etc. — can be justified for ease of use.
The worst part about optimizing for ease of use is that it doesn’t matter if a user can use a feature easily if they don’t want to use it in the first place. The longer we spend optimizing for ease of use, the longer it takes us to learn if this is a feature our users actually want.
We needed to break this pattern. We needed to mitigate the perceived risk associated with releasing a less-than-perfect feature. Fortunately for us, Beta isn’t the first letter in the Greek alphabet.
Alpha AKA “Burlap”
We decided to think about the Beta release as a product of the future, and instead rebrand the first release as Alpha. We found 5 users who had been asking for the Transaction Tagging functionality and “invited” them into our Alpha release. We looked at the backlog of tickets and started talking about them in terms of Alpha and Beta. To strengthen the paradigm, we started describing this first release as a burlap sack: you can wear it as a shirt, but it’ll look dumb and probably be a bit itchy. In meetings we would cue up “Let it Go” from Frozen and play it whenever anyone was worrying too much about the details.
The change was dramatic. Armed with permission to ship an “imperfect” feature, we started to rip through tickets. Alpha forced us to prioritize the most essential elements of the feature, which narrowed the scope of our focus, and resulted in less context switching during code review and when picking up new tickets. Our velocity during the following sprint was roughly double what it had been over the previous 3 sprints.
We ended up releasing the Alpha within 2 weeks of starting to call it Alpha. It’s important to note that (despite our “Burlap” battlecry) we shipped very few bugs. Alpha doesn’t mean low quality, it means less complexity.
Back to Agile
An interesting side effect of the Alpha/Beta prioritization process was that the remaining Beta tickets were almost all design tweaks and small UX changes. Alpha had pretty much covered all of the tickets that made the feature usable, if not bulletproof. We then had the option to look at each of the remaining tickets in isolation and determine whether they were important enough to block our Beta release to 600 clients. We were back to iterating instead of building a larger and larger release candidate. This was better for two reasons. First, the QA burden is significantly less for small changes than it is for a large release. Second, we were now able to get real feedback from our users.
I’m happy to say that Transaction Tagging is now released to a Beta group of over 600 clients, and our Sales team have started selling it to new users.
How we’ll use this in the future
Now that we’ve successfully used the Alpha/Beta pattern on one large feature, we’ll test and refine the pattern on our remaining features this quarter. A rough definition of the framework:
Alpha: The simplest possible version of a feature. Prioritize “can they do it” over “can they do it easily”. The goal is to learn whether they want the feature at all. Keep in mind that your Alpha users will likely be early adopters with a vested interest in the success of the feature; take their feedback with a grain of salt.
Beta: Take the time to clean up obvious UX and design deficiencies. Use metrics associated with Alpha to prioritize any remaining ideas for the feature. Implement the ability to monitor the way the feature is being used.
General Availability: Once you’ve learned enough from your Beta users, implement the changes you feel are essential and go live. Now it’s time to iterate.
If you are interested in learning more about Bench Accounting or a career with our Engineering team, then please visit us at https://bench.co/careers/.
