(0:34) If you build a startup without talking to users (like his first project about calendars, about which he knew nothing and was not even a user himself) or just around yourself (like he did for Justin.tv), you can be lucky and it could work out if you represent a lot of people with the exact same need, but most often than not it will not go anywhere.
(2:53) For Justin.tv they had built a lot of great tech for streaming, but didn’t have a user case that would allow them to have a great success, so they decided to pivot and had two options: mobile and gaming. They picked gaming, but while they liked to watch game videos, they were not game broadcasters, so that had to do a lot of user interviews to understand them and acquire that knowledge.
(5:14) Who you talk to is as important as the questions you ask. If they talked to viewers rather than broadcaster they would have gotten a completely different feedback. Talk to the specific type of people you’re building for.
(5:50) When you have new idea, think who your users are, what kind of questions you’d ask them, and where you’d find them. At (12:20) there’s a sample interview with a college student, with questions around how she takes notes today (eg: laptop for text-based notes for classes like History, but pen and paper for classes like Physics where she needs to draw something), what apps she uses today for text-based notes (Google Drive and Evernote) and why (to share with other students and merge content together), what kind of people she collaborates with, how long notes are, what’s the volume of her note-taking etc.
(16:20) Don’t ask about features, don’t start designing the product during the interview already. Rather focus on understanding if the interviewer has a problem with the way she does things currently, and if it’s big enough to be worth tackling.
(18:09) Talk to at least 6–8 people for every potential group (eg: Stanford college students, high-school students etc. in the case of the note taking app).
(21:50) Once you have an idea that you think can solve a problem, you can either implement it straight away if you’re a developer, but if it’s going to take 3 months to do something that it’s worth using, you want to validate the idea before investing that time.
(22:50) Don’t go back to users with your idea is good and if they like it, because they’ll likely say it’s great. But if you then implement it and offer it you might find that they don’t care enough to switch to your product.
(23:50) The best thing to validate the idea is indeed to hack something together, but a version 1 must be the minimal thing that gives some value. Find a way to “cheat”, do it in the easiest way possible, and see if it’s actually useful to people. If you can get people to commit to paying for it, that is the best validation. If a student wouldn’t spend $5 for your app, they’re probably not very excited about it.
(26:12) He shows a slide of real feedback he got before launching Twitch from existing Justing.tv broadcasters. There’s a list of feature requests on the slide, but he didn’t address them, saying that if people keep using your service if means that those problems are not big. Then he shows a slide with feedback about Justin.tv from someone that used a competitor broadcast service, and this is the most important list of thigs to address, because they were big enough for someone to not use their service. They also talked to non-broadcasters to understand why they would not broadcast, because those are the people you need to expand your market. If the problem was slow computers, they did for bought people new computers. If the problem was that broadcasting was too difficult to do, they wrote software for Xbox and PS4 to do live broadcasting with ease.
(31:55) Don’t focus on people’s desired feature list (eg: polls, child accounts etc.), focus on the goal they’re trying to achieve (eg: making money, stability of the platform, global access for their viewers etc.)
(33:16) Analyzing usage with Mixpanel and Google Analytics is good, but it doesn’t tell you where to go and what the problems are, so you would just invest ideas that turn out to bad.
(35:00) A common problem during interviews are showing people your product (you want to get what is in their head, not put things in their head) and talking to who is available rather than talking to who you really need to talk to.
(36:44) To convince the company to build something, actually show them the recorded interviews.
(37:47) Don’t do user interviews via email, because they must be interactive. Use Skype or in-person.
(41:05) On-site user feedback doesn’t tell you what to build, but it’s important when you have a new feature to launch and want to see beforehand how people are going to use it. It’s a completely different type of interview, more similar to what you could get out of analytics tools (but before the feature is released).
(45:22) The best interviews are with users that talk a lot and ramble, because that gives you context and let you understand what are the problems that they want to solve.
