We just overhauled our app in a hurry. Here’s what we learned about scope along the way.
Any time you’re testing a new software idea, it pays to minimize the project’s scope. Making the right cuts takes judgment, especially when you’re making major changes to an established product.
Minimum Viable Product: A Crash Course
If you’ve ever worked in (or studied) the world of startups, you probably know the term “minimum viable product”, or MVP. If so, feel free to skip this paragraph. For the uninitiated, MVP is a concept popularized by startup guru Eric Reis, which rests on a basic principle: if you want to create something people find valuable, you’re much more likely to succeed by basing your decisions on insights gained by studying real customer behavior. Unlocking those insights as soon as possible therefore requires delivering a version of your product in as little time (and with as little effort) as possible, provided it allows you to start measuring and learning.
Shielding a product for too long — adding all of the ‘necessary’ features, waiting until it’s ‘ready’, before releasing it — delays discovery of critical insights that lead to the right decisions, often until it’s too late. Of course, there’s a flip-side. There’s real danger in releasing a half-baked product before it’s ready to serve its purpose.
It calls to mind an episode of Arrested Development , where GOB promises to build a new house in 2 weeks, despite Michael’s insistence that it can’t be done in anything less than 2 months.
The Bluth family scrambles to get the job done in time for the ribbon-cutting ceremony in front of the board, and they somehow actually manage to build something that passes for a house. It’s really just a facade, but it’s enough to fool the board — that is, until GOB cuts the ribbon and the walls come crashing down.
The MVP is often seen as a tool to test fundamental business hypotheses on early adopters — a set of users who are so excited about a solution that they won’t really mind if the walls aren’t exactly sturdy yet. But the Bluths were in a different situation entirely. They built a prototype (and a weak one at that), when the board was expecting a real, quality home. When what was delivered fell flat, it was a real issue.
All this is to say that just how “minimum” you can be depends on the context and the expectations of the people who you’re rolling something out to.
Old Users, New MVP
The startup I work at, Axial, is not exactly young. We’ve raised 3 rounds of funding, and we serve a user base that we’ve spent 7 years cultivating. This past June, we completely overhauled the core of our application, in a project referred to internally as “Deals 2.0”. We set (and hit) an ambitious deadline, but not without cost. Like the Bluths, we took lots of shortcuts, forgoing a bunch of “no-brainer” features which our users had come to rely on and expect. Granted, we had different reasons for our shortcuts (which had nothing to do with deceiving our board): since we were really tackling the foundation of our application, a lot of things were going to break along the way — more things than we had time to rebuild all at once. As you can imagine, this led to a fair amount of anger, confusion, and pain.
Several months later, we’ve had the chance to smooth over most of those deficiencies with a steady cadence of “fast-follow” releases. Now that things are calmer, and we’ve had some time to measure and learn about Deals 2.0’s impact, we can reflect and draw lessons about the process: Did we take the right approach? Were the initial shortcomings of “Version 1” worth it? If we could do it over again, what would we change?
Why We Did It
Our product was flawed. It was obvious in early 2017 that our product had several fundamental issues. The technology was tangled, which slowed development, the UX was unintuitive, the way we approached our customers’ problems needed refinement, and years of best-intentioned choices had left us with a list of shortcomings, failures, and contradictory features that sometimes did us more harm than good.
Armed with years of knowledge about what works and what does not, and, like all startups, a not-unlimited amount of runway, it was clear that if we wanted to re-engage dormant users, acquire higher end customers, and have a shot at getting anywhere near critical mass in our marketplace, we needed to re-calibrate the core of our app, and fast.
Urgency. We had just emerged from a major tech refactoring which translated to 8 months without much new user-facing value beyond cosmetic updates. Areas of the business needed a boost, morale was impacted, and Product and Engineering needed to prove to ourselves, the rest of the company, and the board that we were now in the position to deliver a sophisticated product in an impressively short time.
Efficiency gains. We had our sights set on continuous deployment, instead of the big-bang releases which had been the norm. Instead of releasing once every few weeks, we wanted to be able to ship every day, to iterate in a truly agile way.
How We Did It
In the weeks leading up to the project’s kick-off, we scoped it out based on a combination of research and intuition. Then we took that scope and cut it in half. I winced as we slashed basic affordances left and right, hacking away at our beautiful concept until we had what seemed like the true, barebones MVP. And as the deadline grew closer and closer, we continued to cut, just to get it out the door.
What We Got
What we ended up with when we released Deals 2.0 was an improvement — from some angles. But what was there was often glitchy and slow, and so much was missing. Some really basic things. To be honest, shepherding this product through the “fast follow” period was often embarrassing and painful.
I had account managers coming over to me as many as 5, 6 times a day — reporting issues, asking for workarounds, asking if this or that was possible, when we thought a feature might be done. This was every day of my life for about a month and a half, until we got the bulk of the critical features taken care of and the pain subsided.
Lo and Behold: Users Weren’t Thrilled
If you’re thinking about about doing something similar with your product, and you’re wondering how your users might react, I can save you some of the guesswork. Here’s some of the worst feedback we got from our users as a result of delivering this MVP:
“I must say I am not at all impressed by the new system and to be quite honest I am disappointed. I have been using Axial for over 4 years now and was always happy with its simplicity and user friendly features. But this newest version has gone away from everything I liked about the system and I can’t see a single benefit from the upgrades.”
“Not sure what happened with the updated interface but it seems functionality is diminished significantly — the update is fairly disappointing.”
“Frankly; my frustration level is 11 on a scale of 10. If I am unable to access Axial by the end of business today, I will begin searching for another platform.”
“If this is the standard of the new functionality, I think we will need to utilize a more user-friendly platform.”
“How do I change the status of a deal — I am being inundated with emails for a deal that is under exclusivity. The only option seems to be “archiving” the deal which makes it invisible to me? This is not exactly convenient.”
“The ability to delete or archive a deal seems like the most basic feature I should be able to have. Please get these features fixed immediately or go back to the old system that worked, seems you rolled out a faulty product.”
But Believe It Or Not… We Were Achieving Our Goals
I’m happy to say it wasn’t all bad. It laid a groundwork which aligned more closely with what our users really wanted, and displayed unprecedented levels of sophistication about the space we’re in. The big strategic shifts we committed to were already paying off with prospects and dormant users. Also worth noting that the app was significantly more intuitive than before, and more beautiful too.
An account manager re-engaging a user said, “I explained the new focus and product changes, which they thought aligned well with their process. Just demo’d them yesterday and they shipped their first deal today.”
And: “I walked a main user through the new updated platform and explained the shift in focus… He was extremely happy with it and repeatedly said how much of an improvement it was.”
New users said: “This tool is absolutely terrific. We love it.”
And: “We’re slapping ourselves on the head for not using this sooner.”
And: “Thank you so much, this is really great. I don’t think you guys could have made this any easier to use.”
Lessons Learned
The pain our users reported did have a real impact on our metrics. Users don’t want to use an application that is unreliable or seems to waste their time. Some users wouldn’t even use the app until we rebuilt the features they viewed as critical.
After the release, we were forced to be reactive to our user’s most critical pain points, scrambling to put the puzzle together well enough for them again. It’s true that by working like this, we did discover that some of the things we had thought would be “critical” turned out to be far from it. But of course, since we had a historical legacy to lean upon for learnings, many of our “hunches” were spot on.
Another plus is that we’re now in a position with several months’ worth of data from the “new world” which is enabling us to analyze and brainstorm how to optimize in novel and exciting ways.
The numbers have largely recovered, but frankly, I’m not sure that I would have cut quite quite so much if I had the chance to do it again. Deals 2.0 solved a lot of lurking, systemic problems, but many of those were invisible to users, and from their eyes the release created a great deal more problems than it solved. And that matters, especially when a big motivator for the project was improving brand perceptions.
The conventional wisdom when defining an MVP’s scope is: when in doubt, simplify.
To that, I would add a word of caution: if it feels too simple, it may be too simple. If you think you’re about to release something that’s going to make your users angry and which could have significant negative impacts on usage and even revenue, do a gut check. Is it worth it to hit your deadline, or could you save yourself, your business, and your users a lot of pain by pushing the project back a few weeks and getting a few more of the basics out of the way in advance?
