Sunk Cost Sinks Ships
Product Management is in. More and more, you hear stories of making that one critical decision that took a product from 0 to 100; the underdog tale of some scrappy startup founder solving for that perfect problem; that single insight that changed everything. This post is not about that.
This post is about what isn’t discussed much. It’s about the products that go wrong. Not the catastrophic, risk-the-company type either. It’s not a sexy post. It’s about that product you all know about, lurking in the background of your company and sucking up time and skilled people that could make a difference somewhere else.
It’s about products without focus, without a story — the ones you forget about until someone reminds you it’s still around. It’s the type that’s hardest to kill but least likely to truly make a difference; it’s the ones that create the fascinating dilemma of sunk cost. I’ll help you identify these types of projects, feel confident to step back to reconsider, and ultimately make the right decision for your company.
I worked on one of these projects at Shopify: here’s the story of what I learned and how you can avoid the mistakes we made. I’ll tell the story in six parts:
- What we were building
- Deciding our initial priorities
- Creating the building blocks
- Testing our assumptions
- Making a hard choice
- Empowering our ecosystem
1. What we were building
The Mobile Store Builder (MSB) was one of the biggest and most difficult things built at Shopify — having been started in late 2015 and worked on by several different teams, people, and across three cities (and even renamed a few times!). We presented it at our developer conference Unite and held many user testing sessions with our merchants. We had a lot of professional (and personal) skin in the game. It’d be embarrassing if we failed.
MSB was a fantastically ambitious project with the goal of building native mobile iOS and Android apps for our merchants. Their customers would be able to shop through fluid, native experiences that could leverage native-only features like the camera, push notifications, and AR.
Obviously, this isn’t something that could be done all at once. So, when we had the vision for this mobile-first future in 2015, we got started on the initial building blocks to get us there.
At Shopify, we started looking at this when we were starting to truly ramp up our mobile efforts, knowing that we were growing globally — into more and more countries that were mobile-only or mobile-first. Massive improvements to the Shopify Mobile iOS and Android apps were the first steps to this for store management, so it made sense for our next strategic investment to be in native mobile commerce.
2. Deciding our initial priorities
Our first Unite conference in 2016 was the first time we talked publicly about the building blocks we were creating: the Mobile Buy SDKs for iOS and Android. These were open source SDKs anyone could use to power native apps for merchants on our platform. By creating these and hinting towards the future — flexible, dynamic components that would eventually power our concept of MSB themes — we saw strong initial interest in this mobile-first future of commerce. We even saw Hodinkee receive a $52k order placed via Apple Pay using a Mobile Buy SDK-powered app!
Once we had these pieces in place, we started to think more deeply about the space. What will commerce look like in 2020? 2030? What can we do to be the forerunner in this area and not only enable our merchants to do better, but allow us to learn more as well? This would only make our feedback cycle faster, giving us better and better insights to further entrench Shopify’s place as the power behind the future of commerce.
With the benefit of hindsight we can see this was a great first step. It was a necessary step for us to build in this space and our Mobile Buy team at the time had the foresight to open source the outcome so our entire ecosystem could prosper.
3. Creating the building blocks
Now that the fundamental pieces were in place to more easily create native iOS and Android apps powered by Shopify, we started to look more deeply into creating apps themselves. We spent time diving into what made a commerce app great: what drove you to download it, purchase through it, and keep it installed on your phone. We knew we needed a simple, usable design generic enough to have components and elements shared across many apps, but also customizable to feel true to the merchant and not templated or cheap.
This led to tradeoffs along the way — do we build something that’s great for an individual merchant so we can ship an app out and gather some early learning? Or, do we make it more generic, ship several apps, and delay our time to learn?
Ultimately, we went down the first path and prioritized reducing our time to learn. We released an iOS app for a specific Shopify Plus merchant and spent time learning from both them and their customers. At first, this app was super well received — the merchant and their customers loved it! This brought us to the end of 2016 and we started working on defining and gathering our ideal merchants for the next phase.
As we closed out 2016, we felt confident that merchants truly wanted native mobile apps from the influx of requests we had from our merchants. We had tested an app out in the wild and it was doing well — generating a significant amount of revenue for that merchant. Plus, other merchants were paying tens of thousands of dollars to build and maintain custom apps. Some of those apps were doing really well too — we were going to make their lives much better!
4. Testing our assumptions
The beginning of 2017 was spent researching mobile app best practices, pitching to interested merchants, and building out more generic features and components. This is where we really started to see problems sprout up.
Our single app experiment had gone well, which caused our team to underestimate the amount of work to build, maintain, and update thousands, hundreds, or even a dozen apps. Just automating the build process of apps was hard enough! We discovered more and more that delivering the value we wanted was still many months away — our apps were still built by hand and merchants had no method of updating them on the fly: an App Store update was required.
Our team decided to take a hard look at the experience a merchant would go through to request an app and become one of our potential beta testers. For the first half of 2017, that meant filling out a long, 7-page Google Form that required a ton of work on behalf of the merchant. It was long, confusing, and many merchants gave up.
This gave us our first real, tangible goal: create a web interface that allowed you to do everything you needed in order to create an app, then we’d handle the manual work to actually build based on your options. This interface would include a preview functionality so merchants could know what their app would look like — and allow us to quickly validate if what we were building was truly valuable.
We tried out this new interface with a handpicked few merchants from our list of dozens that were interested in an early beta. We gave them access to the builder, guided them along the way, and even did some ad-hoc design work for them to help it move along. The results were far from what we expected.
Nobody got through the full submission process.
The reasons differed merchant to merchant, but there was a consistent theme: “I don’t have time to dedicate to this.”
After a while, we realized that really meant “this isn’t important enough to me.” This pushed us back to the drawing board — why don’t merchants care nearly as much about this as we expected?
Our team dove deep into ecommerce based mobile apps — ones using Shopify or any other platform. In doing so we started to see a trend: the apps that did really well were by brands in the top 100 (Zara, Forever 21) whereas DNVBs and SMBs couldn’t invest in those ways and their apps were suffering for it. Some were even abandoning their native iOS and Android apps and going back to web-only! We looked further into our data at Shopify and saw similar trends: there were exceptions to the rule to be sure, but the brands that did the best had a strong mix of: high-value daily content, quick inventory turnover via methods like flash sales, significant percentages of repeat buyers, massive brand followings on social media, and unique native-only features. This created large amounts of overhead that was usually only addressed by the largest of brands that could afford the effort. We were building for our Plus merchants to be sure, but was this something only for a small slice of that market?
Here’s where we really started to see that we might be on the wrong track. Each italicized line in this section was a strong signal that we weren’t building the right thing for the right merchants. We had tons of work in front of us to get this really out there, but nobody wanted what we were building. They liked the idea of an app, but once we gave them one they didn’t really want or need it. Something had to be done.
5. Making a hard choice
Of all the work that went into the MSB product, this phase was the most difficult. We saw we were able to build what people said they wanted but, once offered, they didn’t bite. We’d spent a year and a half on this and shipped a valuable Mobile SDK in its 3rd iteration, but didn’t really have much else out there. We felt the pressure that we needed to deliver our value and keep the project going.
But we didn’t. We paused to collect our thoughts, anecdotes, and data. We took a hard look at what we were building — regardless of the time spent on it.
Taking this step back was hard to do, but it was necessary. The decision to pause here came from the collective uncertainty of our product team and leadership around the future of the product. Our R&D team went through an annual review, which forced us to step back and reconsider if everyone in the organization was working on the most valuable product. We knew we had to decide to double down if we wanted to release something great — but was that the right choice?
Sunk cost is hard to ignore, but it’s the opportunity cost that really matters.
What else were we not doing because we were working on this? Also, were the assumptions we made when we started this product still true? Was there still a huge advantage to building native iOS and Android apps for ecommerce and were we the best suited to helping merchants?
A lot had changed since 2015 and 2017 in regards to mobile web. Take the checkout as an example: Apple Pay on web released with Shopify as a launch partner, making the browser experience faster and better, Shopify Pay arrived to make checkouts blazing fast, and we released Google Autocomplete to make address forms simple. All of Shopify’s themes became more and more mobile-optimized, with some being designed mobile-first. Browsers and phones became better and more powerful — an app was no longer necessary for a great mobile experience.
Shopify also had two massive hits from our Garage team on the go: Arrive and Frenzy. Additionally, our Shopify Mobile app kicked off projects with the ambitious goal of allowing you to do everything on your store within the native iOS and Android apps — delivering value to the hundreds of thousands of merchants that use it.
We also saw some massive interest from our Partner ecosystem — savvy entrepreneurs and established businesses alike were creating ways to provide interested merchants with native mobile apps via our App Store. They were making some great solutions and had satisfied merchants at just about every level of complexity.
For all these reasons, we decided to stop working on the Mobile Store Builder. The team realized the opportunities we could give were best in other places, and moved on to refocus efforts. It was a difficult but necessary decision, and one that’s paid off with the refocused efforts on new things and partnerships with forward-thinkers in our ecosystem.
When making decisions like this it’s rarely, if ever, a clear choice. I, the product’s tech lead, and UX lead spent long hours debating what we should do. By being open and vulnerable with each other and knowing that we needed to make the best decision for Shopify we were able to make a hard call. All of us went back and forth at different times, but together we were able to come to a consensus that we needed to end this product.
6. Empowering our ecosystem
The sunsetting of our MSB product allowed us to open up different doors to create new opportunities for merchants. Shopify’s fantastic ecosystem of 2,300+ apps and even larger group of Partners had started to tackle the opportunities for commerce-focused native mobile apps. When we started this product in 2015 there were none and then — in 2017, we had nearly a dozen companies focused purely on this space.
Once we had made the decision to end MSB we knew there was a void we needed to fill for our merchants. Despite the fact that we didn’t want to invest further in this product, we wanted to ensure our partner ecosystem was caring for this space for our merchants — and empowering them to do so successfully.
Our team started to do our due diligence on these companies: who was solving this problem well? Who had built something that met the quality and support standards our merchants expected? Most of all: who had a strong vision for the future and would be willing to work closely with Shopify to experiment and build a better future of commerce?
We ultimately formed tighter partnerships with companies in Shopify’s app ecosystem with the ability to address the needs of our largest merchants. These companies impressed us with their quality and vision, which made us comfortable recommending them as a solution to interested merchants. We worked together with these companies to hand over the learnings we had gained throughout our efforts and built new features to make their merchant onboarding experiences better. The end result: our ecosystem benefitted, merchants benefitted, and we refocused on what we were best at.
Finding the right company to partner with isn’t easy. These discussions were ongoing with our partnership teams for a long time before we felt highly confident that they were the right path to go down. We used three criteria to choose the right ones: quality of solution, vision of the future, and willingness to work together. Quality of solution and willingness to work together were ones we could determine really quickly — whereas long term vision was hard to determine. In the new age of interactive apps, a measure we used was how they were thinking of integrating new cutting-edge technologies. These conversations were super productive and should lead to some great experiences in the future.
Warning signs
We’ve gone in depth on the product history of the Mobile Store Builder — what worked, what didn't, and what the end result was. The ultimate story is the success of our product in phases 1–3 skewed our perception as we moved into the 4th and 5th phases. When things became harder and more uncertain we should have taken another look at whether or not we were building the right solution to the problem.
Over the period we worked on MSB, we also weren’t keeping in tune with what was going on outside of us. The environment had changed! Mobile commerce didn’t require native apps to create a great experience now that the mobile web had caught up significantly. Our ecosystem was addressing this! While we were aware that others were looking at the same problem, we didn’t really consider that they might be better suited to solve the problem. Instead, we were focused on the impact that our release would have on their companies — would we put them out of business? We hadn’t even shipped 5 apps yet, and we thought that?!
Since we didn’t account for the above things, the Mobile Store Builder product was kept afloat due to sunk cost. We had sunk time, money, and Unite presentations into it. There’s no way we couldn’t ship it!
Ah, but we could decide to do just that. Sunk cost is a powerful motivator and one that’s hard to be aware of in the moment. It’s only when you step back, breathe, and take a look around you that you can be aware of what’s happening. It took our team too long to understand that but, now that we’ve been through this experience, we’ll know how to see the warning signs, remove ourselves from the day-to-day, and figure out if we’re still doing the right thing.
So, to anyone working on a long or complex problem with inevitable sunk costs:
- Reserve time to ensure the problem you’re solving still exists, with the assumptions you’re working with.
- Take a moment to challenge your assumptions and potential actions.
- Weigh them and consider the opportunity cost of your decision.
- Talk to the people involved — both those internal and external to your project.
Only once you’ve built the distance and context to look critically at what you’re doing can you know if the path you’re on is the right one.
Check out our reiterate series to learn more about being a PM at Shopify.
Interested in taking on hard problems with great teams? Send me a message on Twitter, LinkedIn, or apply directly to our Product Management team at Shopify!
