How to NOT launch a startup
Our horrible 2018 beta launch.
“Of all the forms of wisdom, hindsight is by general consent the least merciful, the most unforgiving.”
John Fletcher
In 2018 we did our small, invite-only beta launch. It was the first time we ever did something like that, and of course, we walked into it blissfully inexperienced. The excitement, the stress, and stage-fright of finally pushing the product we’ve been working on to the people who signed up to our smoke tests was a fantastic rush. It ended up being one of the most enriching experiences our team ever had. We failed. Hard.
Instead of folding and disappearing into the darkness, we did our post-mortem, learned and kept going. The product failed, not the idea; our approach failed, not the concept. “ What do we do when we fall? We pick ourselves up.” And that’s what we did. We came out with a stronger team. It’s more difficult to kill us now, we just refuse to die.
Here’s a short story of how NOT to launch a startup.
“I occasionally meet founders who seem to believe startups are projectiles rather than powered aircraft (…)”
Paul Graham
The road here
It’s March 2018 and our to-do list is almost empty. Two solid years of designing, coding, good days and bad days. We turned a few drawings into a fully functioning music web app. Music was playing through it, loops syncing… this is perfect.
The interface is exactly what we need. I pushed hard for that design. I thought it was perfect. My background is in economics, I’m color-blind, and awful at drawing. Naturally, I’m the best-suited person for the job.
We had thoroughly tested the idea. The first round of Smoke Tests helped us to validate the idea — before a single line of code or design meeting ever happened. The response was overwhelming: people wanted this. That was the initial spark that jump-started this madness. We lined up the core working team and plunged into work. It took time, more than we ever thought it would take. I wish I could say we are a team that goes from idea to launch in 3 days. We’re not. Most teams aren’t. We’re no exception.
A month before, in February 2018, we took the app to the rehearsal studio and played with it (we’re all musicians). It worked. Did some stress tests. Worked. The browser can hold multiple high-quality audio files streaming at the same time. This works!
We got an extra batch of loops from friends and users who subscribed to the smoke test, we ran a Facebook campaign to get more people for the launch. We had music, users lined up. Just a few more touches and we’d be good to go.
We coded a basic admin interface, an email-based invite system and our social media was ready with content. We were getting ready to launch a rocket to the moon. And so, by early March, we could see the light at the end of the tunnel.
L-Day (launch-day) was set for April 20th, 2018.
Anxiety killed the cat
We were getting anxious. Finally, we get to show this to the world! I was bursting at the seams. I was very, very anxious. I was so anxious, I bought a ticket to Amsterdam (we’re a remote team, Federico, our CTO, lives in Leiden, The Netherlands) without much thought — “I’m flying over there to launch this, is going to be awesome!”. Was it my wedding anniversary around the same time I was supposed to be in Amsterdam? Yes. Was it Federico’s anniversary also around the same days I was crashing at their place? Yes. Did I think about it when I bought the ticket? Nope.
During that final month we did the migration to Amazon S3, did a backend cleanup, tested the invite system, our admin system, a basic analytics Federico built and we were — almost — good to go.
Spoiler alert: did I mention anything about thorough QA? No, I didn’t. Did I mention anything about UX testing for the interface? I think I didn’t. Any hints of where this is going?
My ticket to Amsterdam was for April 19th, arriving on the 20th. A 90-minute flight from St. Louis, MO (where I live) to St Paul, MN; a 3 and-a-half hour layover and then 8 hours to Amsterdam. Almost 13 hours of travel time. The same day you decide to launch. This will work, of course.
Fear and loathing in the sky
April 19th came. I packed, said goodbye to my wife and got on an Uber to the airport. Not before I made sure to send the first batch of “Welcome to OneMillionLoops, here’s the link to log in” emails to some users. I wanted to “test” how that went, see what kind of response we’d get and iterate — on my way to the airport and flying over the Atlantic!
For the following 20 hours, I behaved like a maniac. Got to the airport, checked in, went to the gate, double espresso, opened the computer and kept going. Re-read a few Paul Graham essays, my notes on 2 books on launching products and previous notes I made. Got to St. Paul, MN; another double espresso and listed all the early users, friends and family who would get the email.
“Group 5 for Amsterdam Schiphol is now boarding”. Closed the computer, got on the plane, opened the small table, opened the computer and started to re-write the templates for the invite emails. “Sir, you have to close the computer and your table, we’re about to take off”… Sh*t… “We have reached 10,000 feet, it is safe now to use your electronic devices”. Opened the computer, finished the template, planned the next few days, wrote templates for all the follow-up emails, got my notes and the flow of actions for each scenario. 8-hour flight. Not a single minute of sleep. I’m on a roll.
Landed in Schiphol, met Federico, got on a train, arrived at his place, said hello to his wife, fresh pot of coffee, opened the computer, sent 223 invite emails + follow-ups, ate an entire tray of lasagna together with Federico, kept going for another hour, went out for beers.
System crashed. Please stand by…
April 21st, 2018. Noon. Wake up. Feel I’ve been hit by a train. I need coffee and a decent breakfast.
I check my emails and there’s not a lot happening. Strange… OK… we check the app analytics, a good number of sessions but not as many as users on the database. Not even half. OK, maybe it’s early. Let’s wait a few more hours. Early evening comes by, let’s apply the follow-up flow of “hey did you get our invite?” and for those who logged in, ask for feedback. Emails sent. Let’s wait.
And the answers started to roll in…
“None of the links have worked in the emails I’ve received… the invitation, the email login confirmation or when I downloaded a drum loop”
“I first tried on my phone and it was quite hard to navigate, there was an odd control issue that happened whenever the screen rotated. like it would get the horizontal and vertical screen controls mixed up or something. “
“I was trying to access it from an android smartphone and the page was really unresponsive.. in fact I couldn’t get anything to work.”
And a lot more of the like. This is not looking good.
“Sir, we’re losing her”
There were multiple issues that were being reported, and given the time difference between the US (where most of our initial users were) and Europe, there we were… pulling hotfixes like maniacs at 2 AM.
When you logged in to OneMillionLoops.com, we used to send you to the “Profile” view, where you had to fill your personal info (name, instruments you played, your social media, etc). We realized that a lot of users were staying in that view and not doing anything, which was very strange. And then, we got an email with a video.
Sh*t, sh*t, sh*t.
What followed were two weeks of fantastic feedback (fantastic = you’re doing it all wrong). We fixed all (or nearly all) the bugs and let users know when they were fixed. But, there was still one problem that needed more time to fix: the interface was horrible. The same user who sent the video was kind (and awesome) enough to — instead of sending a note -, he actually sent an audio explaining why OneMillionLoops was not working for him.
And that was the case with many others. The interface wasn’t good. Users were confused.
A month after the launch our engagement was almost nonexistent. It didn’t matter how much we pushed, nothing was happening.
Launch: failed.
We learned. A lot.
We let a few days go by, but we felt strong. I don’t know why. I was brain-fried (no surprise there), but still eager to understand, learn and keep going. The team was the same. And so, we did a post mortem. An honest, as unbiased as possible, exercise to understand our next steps.
Below, the main points we arrived at:
What worked:
- We managed to generate demand and users to the application and the idea; this validates that we are on the right idea.
- We managed to generate feedback from users. This was vital to understand our failures.
- We developed an analytics system that allows us to quickly understand what’s not working.
- We understood very quickly that things had to change. Within a month of the launch, we were working on the redesign. This means we are being agile to understand the problems and addressing them.
- We reached 65% of the beta users' goal, and although the beta process is not over, we had an interface that didn’t help much. It was a good number to start.
- We managed to implement all the features we set out to do, with very efficient code and excellent browser load. That’s important.
- Users sent us music. We did not have to persuade people with money, gifts or other methods.
- Contacting users through email gives them the initial push to access the app. For a small launch, it is effective enough.
What did not work:
- We did not achieve our user goal, recurrence, and engagement.
- Based on user feedback, the interface was not intuitive. Only 1 out of 65 users had a recurrence and solid engagement.
- Only 1 person used the feedback tab that we implemented.
- 30% got trapped in the “profile” view, which led many to not return.
- We launched without doing QA or usability testing. The rush to launch blinded us of the obvious.
- The services offered were too many: Jam, Sell, Connect. We communicated everything together and tried to cover everything. It was confusing for many users.
What we learned and have to change for the next version
- The interface has to be easy to understand.
- We have to simplify things to the fullest. From the offer to the UX, to the communication.
- QA Testing (cross-browser and cross-device).
- We have to do UX testing with users several times and fix things before launching again until we see that the application makes sense. Let’s be super critical of what we do.
- More personal contact with users. While we were sending emails, perhaps we lacked more direct contact, which prevented us to understand the context and use cases.
- Launch small and incrementally add users. Let’s never again launch to everybody at the same time, especially if we have not thoroughly tested.
There’s a limit to the “launch fast and iterate”. If you frustrate your users and they hate the experience, then there’s nothing much to iterate. They’re gone.
We’re not dead. We’re more alive than ever.
This experience could have easily killed our team. In fact, an experience like this killed the previous team that Federico and I formed a decade ago. What is most surprising to me is the fact that I repeated mistakes, especially launching like this and doing exactly the opposite of what I should have done: be calm, collected and launch in the most efficient way. A decade ago, we launched a product in some similar fashion — although it wasn’t a code-based product — and I (we) crashed and burned. I few weeks after this 2018 launch I felt like a complete idiot. It’s not a nice feeling to have. Never again.
My burnout lasted for a few months, and the team rallied and pushed through like spartans. We want to keep working together. The product failed, the idea didn’t. Our approach failed, not the concept. But we had to change a lot and learn. We changed the interface, we now do thorough QAs, UX tests with users. We don’t take anything for granted.
But, most importantly, we’re creating a process for launching products. We still have a lot to learn, but we’re getting closer. We’ve made tons of mistakes, and we’ll keep making them. If we’re making mistakes, we’re moving forward.
This experience didn’t kill us. It made us a stronger team. In fact, as of January 2020, we are doing our second launch, incorporating 100% of the lessons we learned. So far, it is going quite smoothly. And while things took longer than expected (don’t they always?), we’re proud and happy with what we’ve learned and achieved over the past two years. We’re here. Up & running.
Cherish your failures, they are amazing learning experiences!
Originally published at https://www.onemillionloops.com on January 31, 2020.
