The hidden costs of delivering customer worthy software
“It just works” — if only things were that simple when building software.
You really want to get the decision right if it’s for all eternity, right? You don’t want to pick the wrong mutual fund, or the wrong salad dressing … even if we manage to overcome the paralysis and make a choice, we end up less satisfied with the result of the choice than we would be if we had fewer options to choose from.
It’s easy to imagine you could have made a different choice that would have been better. And what happens is this imagined alternative induces you to regret the decision you made, and this regret subtracts from the satisfaction you get out of the decision you made, even if it was a good decision.
In the end, learning to accept “good enough” will simplify decision making and increase satisfaction
The tyranny of choice
Typically, in our coverage of ecommerce through this publication, we focus on technical analyses of retail businesses.
What we usually abstract away, however, is that behind our high-level reports are vast troves of data and increasingly complex software systems developed by our engineering team.
And each of these systems are in-turn made up of multiple layers — each requiring hours of deliberation and concerted decision making. By developing a better understanding of the choices being made, the whole process, to derive value from investment, can be streamlined and improved upon.
The aim of this post is to introduce the “engineer’s point of view” when developing software (to power the next generation of ecommerce).
No product left behind
“Track every single product sold online” is a target that we’ve long mulled over, seeing as we are in the business of developing comprehensive product datasets. However, the tagline is easier printed on a poster than implemented in code.
We’ve found that, as in many distributions, the 80/20 rule rears its majestic head in this domain — relationships among ecommerce products — as well. Over the years, we’ve often had clients request for long-tail coverage of obscure websites at the proof-of-concept stage, only to later realize that their customers were only interested in much smaller subsections.
Our teams have come to realize the vanishingly small benefits of going from 99% to 99.99% coverage, a maxim that’s regularly proven true across products and industries.
Quantity vs. the world
The other benefit of not focusing on the quantity angle doggedly, is the invaluable employee time which it frees up. Now, effort can be dedicated to solving other crucial problems such as data quality, user experience and uptime guarantees.
There are only finite time and compute resources available — so choosing to work on the right problem is often more important than working on the hard problem.
Understanding optimization limits also reveals the possible regions of operation — the borders across which trade-offs can fruitfully happen — balancing business goals with engineering constraints.
Just some minor changes
A quick change here and a small fix there, that’s all we ask. Unfortunately, it’s never quick, and never small. And even more unfortunate is that being able to deliver a no (without coming off as inconsiderate), takes far more practice than developing the actual skill being asked. So it is very tempting (and often the case) for engineering teams to embrace lofty projects without much debate or consideration of the larger picture.
So on behalf on development teams all over, I’ll just leave this oddly appropriate xkcd here, for the next time it seems that a trivial change shouldn’t take too long.
Done is better than perfect
Another issue that often plagues development teams is the next feature fallacy. While perfecting your product with all the best bells and whistles might sound like a great plan, it is also very time-consuming.
Often times, it makes more sense to step back and face the hard question — “will adding this feature benefit the bottom line”. We’ve observed that our most successful projects have also been the ones where aims were clearly articulated when starting out.
Talented engineers fall prey to similar reasoning too, often building generalized abstractions when asked to solve specific problems. While developing software, we might convince ourselves of its innate beauty, but I prefer the reasoning that for software, beauty is in the hands of the user.
Fast, good and cheap
The easiest summary to this post might be the Venn diagram below. Balancing customer needs with development is always a choice of two from among “Good”, “Fast” and “Cheap”.
With this post, I hope teams are now better equipped when going into the ring to tackle engineering challenges.
Note: This post was in-part motivated by conversations from Rootconf 2017 — where I had the chance to speak about a specific database challenge that we faced. Links to the video and slides, for those courageous of diving into the deep end.
