Jason Schwartz (Spotify)

Product Manager who specializes in growth.

WTF is Product: If you were to meet someone on the street, someone who has never heard of product management before, how would you describe it to them?

Jason: I usually describe it like building a new store from the ground up. We would start with a plot of land, and we’d have to decide to build a store there. Then we’d look at the demographics of the neighborhood, what other types of stores are there, and what people might be interested in buying that’s currently not available; a need that’s not being fulfilled. Then we’d have to design the store, and figure out how to bring a team together that will make it happen. We would empower architects, designers, and construction workers to turn that idea into reality. Building software follows the same path, and the product manager is ultimately responsible. Not for the project management, but for the decisions on what we are going to build, why are we making those decisions, and making priority and tradeoffs when necessary.

WTF Product: What is your own journey? How did you first start working in product management?

Jason: I was working at an interactive agency and at the time web 2.0 was just getting off the ground. I was looking at social networks like MySpace and I had a hunch that brands would start using it to market themselves. I submitted that hypothesis as a panel at SXSW, which at the time was very small. About 2,000 attendees. The word “Social Media” hadn’t been coined yet, and they never had a topic like that at SXSW. It got a lot of interest at the conference, and through that I got in touch with a company called Angelsoft. It was a platform for VCs and investors to manager their dealflow — kind of a CRM for them. That was my first proper product role.

WTF is Product: How do you think product management varies working in small startups versus large organizations?

Jason: When you’re the product manager in a startup, you’re in charge of the entire product and anything that it might do. Every feature request is going to come at you. Prioritization is much more difficult because you are simultaneously resource-constrained and so many people are coming to you with their needs. At a bigger company, everything gets very specialized. At Spotify, I focus on getting people to upgrade from the Free to the Premium tier. I build things very specific to that one slice of the product. There is someone else who is responsible for the Playlist feature, and someone else responsible for Discover Weekly, and someone else responsible for the website. You have very specialized product managers who prioritize as it relates to their mission, as opposed to dealing with requests and priorities for an entire product. That said, you often need multiple product areas to cooperate to achieve a goal. Getting buy-in across a big org is a much more important product skill in a larger org.

WTF is Product: When you’re in these different settings, how does it impact your product thinking?

Jason: When I’m in the startup stage — and this depends on where I am in the startup lifecycle — I likely haven’t nailed my product/market fit. A lot of my product thinking is outward-facing — I go out and talk to people to learn about user needs. I try to learn why a product isn’t resonating, or why a certain feature is unexpectedly doing well. I typically don’t have enough data to just look at metrics.

At a scaled company, I still want to talk to users to figure out what they’re doing, but product/market fit has already been established. I know this is a thing users want and so I’m more focused on optimizing. I can create a test and roll it out to a very small percentage of users and get statistical significance inside of a month. I can find out what users want and don’t want through metrics, which is a very different mental model.

WTF is Product: Would you define yourself more as a gut product manager or data product manager?

Jason: I use data to point me at which things I should be focusing on. Then I use my gut to tell me what we should be building to most effectively move the target metric.

WTF is Product: Do you have stories or examples of products you’ve launched where you got a really unexpected outcome?

Jason: I had an app called Matchbook which was for remembering places. The idea came from being in the daily deals space. We were looking at which things correlated with a good deal. This seems like an easy thing, but when we tried to tell a sales force the criteria for a good deal — it was really, really tough. What we discovered is that people would respond favorably to a deal if we had an indication that they were already interested in going to that place. This isn’t a mindblowing insight, but it was the only thing we could find, so we tried to find a source for this data.

It turns out that a large percentage of people have a list of restaurants that they want to go to in the notepad of their phone. If you ask those people, “Do you use the list?” They say, “No. I don’t know where these places are or why I added them.” But then, if you ask, “Do you keep adding to the list?” they say, “Oh Yes. I’d feel terrible if I didn’t. Like I lost the place.” This is a pretty solvable technology problem, so we built an app around it. You put the places into Matchbook, and it organized them by neighborhood and on a map so you could see where the places are. At the time FourSquare was huge and we figured people would want to see what their friends were saving. It turns out that’s not what people wanted at all. Their notepad was a very personal and private list. They didn’t want to share, and that’s why they weren’t using some of the more social apps. They just wantd a private app for themselves. We learned that by prototyping the app, before we launched.

WTF is Product: What have you seen as the best ways for product managers to collaborate with a team of designers, engineers, etc? How you do you think about the PM role in relationship to other disciplines?

Jason: I’ve tried this a number of ways, and this is highly dependent on the level of engineering expertise. The ideal state is that I frame the challenge and why that challenge is important. I show why solving that challenge will move the metric we’re trying to move. I put it to the team to figure out how to go about doing that. I guide the discussion, but the team is coming to their own conclusions. As a result, they are invested in the product they are about to build. When they are implementing and they come to a crossroads, and they ask themselves, “Should I implement it like this, or like that” they can answer the question for themselves rather than needing to ask me.

This doesn’t work in every case. I’ve definitely worked with teams that didn’t want to be that involved in the process, or they didn’t have enough of the business skills necessary to come up with impactful solutions.

If I’m more prescriptive about what should be built, the result is that the engineers come to me at each decision point with questions. They haven’t internalized the solution enough to be able to answer those questions for themselves. It’s even worse when they don’t come to me at each decision point and we need to re-code things because they went down the wrong path.

WTF is Product: In your current role as product manager for conversion, I’m sure there are specific buttons and various pieces of technology that you focus upon, but it’s really trying to optimize a certain behavior. How do you think about that type of work that is really behavioral? Ultimately, any product work is behavioral, but this is even more so than the classic type of product work one would imagine.

Jason: The way I approach it is by looking at the users who have already converted. There is something that those users have discovered about the app which created an “aha” moment. It allowed them to go from the state of just using the app, to paying for it. I try to tease that moment out. It’s often very difficult, but the data is in there somewhere.

If I can discover the habit path that makes the lightbulb go off, I can focus my product work on getting more people to take that path.

I’m figuring out which user actions are correlated with conversion. I typically have a long list of correlated features that I need to systematically test for causality, and most of them will not be causal. To make matters more complex, the habit path will likely involve getting the user to take an action multiple times, or to take multiple actions in conjunction. This process is not easy, and certainly not quick.

WTF is Product: How much time at minimum do you think you’d need to go through a process like that?

Jason: A year or two. Most companies don’t do this work in the startup stage. They don’t have the time or the data to do it. A lot of companies that reach product/market fit do so by happenstance. They happened to be at the right place at the right time, and happened to build the right thing, and it just worked. It takes years for them to circle back and figure out why it worked.

WTF is Product: Do you have any stories of “aha” moments that you’ve discovered in this type of work? Any time where you’ve made it through and found that sort of causal relationship and then implemented a change in the product, resulting in the change you were looking to see?

Jason: At Matchbook we analyzed what turned someone into a long term user. There was a low retention rate, but those users that retained were die-hard. We looked into what those people were doing, and realized they saved something on their first app open. If they opened the app, thought it was cool, but never created that habit of “I’m going to put things here instead of my notebook,” there was no habit created for using the app. All of the product work became focused on getting them to save a single thing on their first app open. The app wouldn’t let you do anything else until you saved a place. After that change, we watched the retention curve go way up as more people got themselves into that habit path.

WTF is Product: How long did it take to realize this behavior?

Jason: I was already a year into the product, and it took another three months to run the experiments that moved the needle.

WTF is Product: How do you decide to roll something out as a test vs. rolling it out at 100%? Do you ever decide to roll something out at 100% without testing it?

Jason: At Spotify everything we do is rolled out as a test. We need to make sure that we’ve increased conversion without tanking another metric.

WTF is Product: What do you say to someone that is starting in the product management field, that would be coming in at a junior level into a product team? How should they get started and what growth could look like for them in the discipline itself?

Jason: That’s one that I’ve struggled with. That’s why I started teaching product management at General Assembly. If you ask most product managers, they will be tell you they fell backwards into it in some way. The story is usually that they started something on their own, which made them the de-facto product manager. The other path is they were working at a place which didn’t have product as a discipline. They showed interest so it fell to them to do it.

They probably bumbled through it, as I did for the first couple of years. I figured out the things that worked, and more often the things that don’t. I think it’s really tough to get into. I was thinking about writing a book about product optimization and growth to give people a playbook.

WTF is Product: It’s a very unique discipline in that regard. And it varies from the startup context — where the product manager is often the CEO and there is that entrepreneurial path that you read about a lot. The lean startup methodology is in part a product development methodology as well. But then there is, increasingly as tech has grown as an industry, companies of larger scale have product organizations and the ability to start in an associate product role, then move to product manager, senior product manager, etc. I think other disciplines have a better understanding of what maturity in the field means. What does it mean to start a junior level and grow mature within the discipline?

Jason: It’s something that they’re just starting to codify at Spotify. We’re trying to answer what each of those levels mean and how you’d progress to the next step. I’m not sure I have a good answer on this.

I think that the challenge junior product managers face is they have a lot of people giving them feature requests, and they let that dictate their entire roadmap. They need to say, “I hear your request, but this isn’t the priority.” That’s a big hurdle for junior product managers who are typically pushing back on someone more senior.

The other challenge is that junior product managers think they are there to build product, and they build all sorts of stuff. That’s not a product manager’s role. A product manager is there to guide a team in determining the most effective way to move a metric that’s important to the company.

When people make requests, they can learn to ask, “How does this move our metric?” And even if it does, they can then ask, “Is this the most effective way to move the metric?” Asking those two questions consistently seems to be a steep learning curve. That seems to be the switch for when someone goes from being a more junior-level product person to a senior one.

WTF is Product: Are there products right now that you’re obsessed with? Are there products right now that you’re using that really excite you and for what reasons?

Jason: I feel like I used to download every new app to see what they built, but I do that less and less. I just got a new phone and there must have been 10 pages of apps on the old one. I’ve only downloaded one page — I clearly didn’t find a lot of value in most of them. I didn’t look at my old phone, I just downloaded the ones off the top of my head.

WTF is Product: Which one was the first one?

Jason: Spotify [Laughs] Probably the next one was an app called SimpleNote. That’s been out forever. It’s super stripped down. It has a tiny fraction of the features of Evernote, which is why I love it. It’s a notetaking app. It syncs on my desktop, it syncs on my phone. The syncing is perfect. They nailed that one thing and it just works. That’s probably the best app that I have. I feel like that’s the work I’m always doing, stripping things down and making them more simple. This is an app that started that way and I really respect that. It’s literally in the name.