How To Define Your Early-Stage Product Development Process

Steve Huffman (Reddit) and Emmett Shear (Twitch) describe their product development process in the early stages.

leanentrepreneur.com

A few months ago Y Combinator released a video interview of Steve Huffman, founder of Reddit and Hipmunk, and Emmett Shear, founder of Justin.tv which later became Twitch. This video is full of great advice and lessons learned about what to do to avoid costly mistakes down the road.

Startup advice like this is so valuable, that it’s a shame it is not more widely distributed and accessible (That’s why we created Playbook). In attempts to share these lessons with more people here is a slightly edited version a transcript of this interview.

If you want to watch the full video, it’s embedded below at the bottom of this post.

Question 1: How did you talk to users in the early stages?

Steve Huffman (Reddit / Hipmunk)

Every company is very different. It’s important to keep in mind that not everything that works in one business will work in another.

At Reddit, the entire product was talking to users. The challenge then becomes which users to talk to. At Hipmunk it was completely different. We used Olark to talk to users, and it was indispensable.

One lesson we learned is that your angriest customers can often become your most loyal customers. It’s because of the absolute value of emotion.

Emmett Shear (Twitch)

At Justin.TV (early version of Twitch), all of the best product decisions we made, were because we were building the product for ourselves. It’s possible, especially very early to build the product yourself. If you truly are building something for yourself, then you don’t have to talk to users in the beginning. Eventually, you will want to, but for the first three months, you don’t have to.

When talking to users at Twitch, I learned that you could get a much better understanding of your users if you talk face to face or through Skype.

The other thing we did that was very valuable was talking to people on other platforms. If they deliberately chose not to use your product and chose another, then they have a specific opinion about why they didn’t choose your platform, so you know what to work on.

Question 2: Can you guys describe how your MVPs worked in the beginning?

Steve Huffman (Reddit / Hipmunk)

I wrote my first line of code on June 3, 2005, and we launched on June 22. We actually didn’t even plan to launch. Paul Graham just linked to us from his blog without telling me.

At that point, we didn’t even have a plan for Reddit. We sort of just followed the users, repeating to ourselves “what is best for the users and what is best for us.” We gained a lot of loyalty by doing that. That would not have happened if we were sitting in our apartment not launching.

In the next six months, we created a ton of features. Only about 25% of them ended up lasting more than a day or two. There was stuff we thought was a great idea that didn’t work. If we had waited to include them in our launch, we would have waited until November and then wondered why they didn’t work.

Emmett Shear (Twitch)

I think “minimum remarkable product” is a better phrasing than minimum viable product. A viable product brings to mind this whole ecosystem of support like a viable organism, which has all of these things it needs to live. You don’t actually want that. If you’re launching something that is self-sufficient and can go off and live on its own, you definitely waited too long.

This thing is going to be on life support when you launch it, and you are that support.

You just want to get to the point where at least one person in the world finds it remarkably good. You don’t know if you’ve actually done that until you put it in front of other people, that’s why you want to launch it so quickly.

Question 3: How do you incorporate tracking and analytics into building a product?

Steve Huffman (Reddit / Hipmunk)

At Reddit, we just didn’t. We didn’t look at traffic for years and didn’t measure anything. We did product development by intuition, and it ended up working out. It is a huge pain in the ass now.

There are a lot of corners you can cut when you’re building your MVP, but lack of data can haunt you. Think very carefully about what the minimum viable data is. You don’t even have to look at it, just log it somewhere, so you have it when you need it.

If I could just go back in time and give myself some advice, it would be on this topic. I would say just spend half a day researching analytics best practices on collecting events and storing them.

Emmett Shear (Twitch)

You’ll thank yourself later if you have historical baselines for certain user actions. If you don’t have it, it’s awful. You have to wait for 3–6 months to get enough data before you make a decision.

I got the best advice from the guy who runs Mixpanel. He said pick your top 5–7 user actions and just log those. Most of the things people do aren’t important; you can kind of just ignore those.

Another thing I’d recommend is using a third-party service. Services like Google Analytics and Mixpanel are better than what you need right now.

Question 4: What are some tips and tricks to come to a consensus on what to build after you’ve talked to your users and looked at data?

Steve Huffman (Reddit / Hipmunk)

The first thing I say is that I don’t want to argue about it if we can just test it. If it’s not in my top three things to worry about for the day, then there’s not reason to bother. This, however, is easier if you have more resources.

At Reddit, we’ve had disagreements about cross promoting from mobile to desktop. It’s not great for the users, but it does drive revenue. When you are in those situations that’s when disagreements come up, but you’re usually arguing about short term or long term, or what’s best for the business and what’s best for the business. But once you beat at it and beat at it you’ll eventually come to a good solution.

If you are getting emotional about the details, there is usually something fundamentally wrong. You may be swimming upstream, that happens a lot.

Emmett Shear (Twitch)

Firstly, if you find yourself in the situation of “oh I’m going to go talk to my users to validate my product idea”, then you’ve gone horribly off the rails. You do it the other way around. You don’t talk to your users to validate your product idea; you talk to them to get your product idea.

If you haven’t talked to users and you haven’t looked at data, you don’t get to have an opinion about your what to build.

Once everyone has actually looked at the data and actually talked to users it’s pretty rare to find a disagreement. I find it better if you just have one person in charge. You can argue your opinion but having someone in charge helps you move faster. Decision making is far too slow in a startup.

Question 5: What advice can you give about the product development process early stage startups encounter after launching.

Steve Huffman (Reddit / Hipmunk)

In terms of the actual process, it is going to be very different for every company, and it is going to change as you scale, but I do think it’s very important to have a way of working. You want to start out really specific.

The way I like to think about it is: “Where do you want to be in a year? Now draw a straight line from there and think, what do I need to be doing today. Do we need to validate any hypotheses? Probably, so do that. It’s just a straight line.”

I like to work in two-week cycles. So check in every two weeks. If everything is working out keep going, if some assumptions were wrong, adjust your direction.

Emmett Shear (Twitch)

Regularly checking in with users doesn’t mean weekly course corrections. I did a whole bunch of talking to users at the beginning of Twitch, and then, I didn’t, but we could have gone six months without talking to users because we already knew what our users wanted.

You do need to go back and talk to users and learn about some of the nuances, like the things about your service that is bothering them, or if they find something really cool.

The goal of talking to users is not to get them to tell you what features to build. Users are really bad at that. They have no idea what features to build. The goal of talking to users is to get to understand them really well.

Personally, I prefer more in-depth time with my customers and learn a lot about them. Then depending on what we have to ship I may spend six months just looking at metrics and not actually talking to more people.

Question 6: Can you guys walk us through a feature that is on your site today, bad or good, and walk us through how it came to be?

Steve Huffman (Reddit / Hipmunk)

The early days of Reddit was just links. Everything was external. Six months in we added comments.

On any link, you could click on the link, or you could click on the comments to go to the comments web page. Our users found a way to make a post that linked to its own comments page. This is now what is known as self-posts. Self-posts now make up 60% of the content on Reddit.

We would never have built that feature; it didn’t even make sense in terms of what the platform used to be. Even most users didn’t understand this concept.

The best product ideas are the ones that your users are doing anyways, and you just grease it a little bit.

Emmett Shear (Twitch)

All of our users showed us pretty clearly that they wanted to sell advertising and make money off of their streams. They also told us that their least favorite thing about our competitor’s products is that they show ads over the streams and it disrupts the experience. As a product manager, you have to solve for both. You can’t just pick one.

So what we did was give users a button to run ads on their own channel. Because we were giving them a cut of the revenue, they were very incentivized to press that button. We basically had a paid workforce that made us money in the downtime of the stream.

This points to a very important principle in product design in general which is: you want to try to invert all of your assumptions. We had this implicit assumption in our heads that we had to choose where to display the ads. When we gave up that assumption, we could find a much more elegant solution.



Before you go…

If you want to get more startup advice like this check out our mobile app Playbook. With Playbook you can follow all of your favorite startup authors like Y Combinator, Marc Andreessen, Andrew Chen, & more. We’re still working on product market fit so we’d love to know what you think!