Mind the Product: Key Takeaways From the 2018 Conference in San Francisco

Source: Mind the Product

I attended the “Mind The Product” conference in SF last week, along with a smaller session for senior product leaders. Given the investment of both time and money that I (and my company) made in this conference, I’ve tried to synthesize my key takeaways to benefit my team. Posting those here in case it helps you.

Here were the highlights for me; the organizers have also posted a summary of the conference here.

‘Mind The Product’ Highlights

1. Ethics in AI / ML and the role of product managers

This was by far the most interesting and thought-provoking series of conversations for me, given how much this topic has been in the news recently and the fact that I am increasingly confronting these questions in my day job working with machine learning (ML). A quote that stood out:

“PMs need to cultivate a new relationship with the data that decides our future”

Mind The Product: A few key takeaways from this set of talks / panels:

• PMs need to understand biases in data used for training models. This example from Boston was particularly compelling. There was another example of a company using essentially a look-a-like model for screening resumes — trying to match resumes with profiles of high performers, not realizing that it was propagating existing biases implicit in their current recruiting process
• Data collection should be effortless and continual e.g. Tesla and their huge array of sensors in the car, which take feedback from human intervention to make the self driving algorithm better
• Understand that these systems are brittle, and can break — the past doesn’t necessarily predict the future
• Think through the worst case scenario in advance
• Ultimately, product people cannot abdicate responsibility for machines doing bad things

2. Feature abandonment

This wasn’t an explicit talk — but came up in multiple different conversations. Feature abandonment obviously makes the product confusing e.g. experience dead ends, features that don’t work etc. It also creates perverse incentives for the product team to wait till they ship something that’s close to “final”/ perfect because they are worried that they will not get a chance to iterate. Important to think about principles of lean start-ups and conduct frequent feature audits where you’re going back and killing features

3. “Don’t call them soft skills!”

Interesting series of talks at ‘Mind the Product’ on how “Product EQ” is a key skill we need to hire for, train and develop. My favorite quote from a panel on this — courtesy of Gibson Biddle:

“How do you get to be a better leader? Be a better human being.”

Validating words for me — because I am often accused of being “too nice”!

4. Managing a product team

• Write a “user story of me” — similar to a “manager read-me” concept that I have been discussing with my team. Good motivation to actually write this.
• Great discussion from Christina Wottke on things like team OKRs / retro’s etc. A special shout out to her on thoughts on the polarizing topic of the weekly email
• Some interesting frameworks for collecting and synthesizing team feedback — new frameworks: Carbon 5 / Spotify health check model

5. Career management

Another gem from Gibson Biddle: build your “personal board of advisors.”

6. How to do research well

Recognize cognitive biases in product and research, and ask questions accordingly. Helpful reminder to not ask “yes or no” questions, but various types of open ended questions to avoid cognitive biases. For example

• Confirmation bias: i.e. we only look for data that confirms an existing hypothesis, and discard data that contradicts it. Application in terms of research questions: instead of asking “how likely would you be to use XYZ”, ask a more open ended question like “tell me about that you’re currently doing in situatioin XYZ”
• Cognitive dissonance: i.e. our mind is incapable of keeping inconsistent thoughts, which leads us to believe in a products success / strengths and gloss over potential weaknesses. To combat this — ask a question like “lets assume <undesirable truth> is true — now what?”. Don’t just talk to power users — also talk to customers that have churned, low usage customers and customers that are using competitor products. Give people the permission to complain (because sometimes research participants are too nice) e.g. “if you have to change one thing about this product, what would it be..”
• Curse of knowledge: “suppose you had a new person join the team, what advise would you give him”
• Social desirability bias: i.e. we edit what we say to make ourselves look good. For example — “how often do you go to the gym” frequently results in untruthful answers. Instead of asking something like “have you ever missed a deadline”, ask “tell me about a time you did something later than originally promised”
• Backfire effect: Presenting rational evidence against our beliefs can make us reject it even more. So instead of saying something like “you’re actually wrong..”, say something like “you’re right — and..”
• If there is no cost to customers, they will say Yes. On one hand — an obvious thing to keep in mind (which is why conjoint analysis exists) but helpful to think about all the same. Ask questions like “since you don’t have XYZ today, what’s your current work-around”, or “if you had XYZ already, how would it make your job / life easier” (and run like hell from product ideas that would be a “nice to have”

Mind the Product: Helpful reminders

Not new nuggets — but helpful refreshers:

• We screw up research — because we don’t have the time
• Staying in the problem space more than the solution space
• Getting users to take the core action e.g. FB / Twitter — following other people
• The idea of virtuous loops: accruing benefits — the more I use this product, the better it gets, mounting loss — the more I use this product, the harder it will be for me to leave. Somewhat similar to the hooked model
• Stakeholder mapping in the first few days of a new job

Book-ending this with my other favorite talks

Being “indistractable” is the new super power — from Nir Eyal, with helpful tips on how to understand and manage internal and external triggers that lead to distraction. Sounded like he is writing a book on this topic as well — won’t spoil it for you.

The best moment was Martin Ericsson (or @BFGMartin) talking about suffering from imposter syndrome despite being incredibly successful in the space. Definitely identified with this thought — it’s sometimes intimidating to be a generalist in a world of specialists (engineers, data scientists, designers), and its helpful to remember that product managers need to be comfortable with being uncomfortable!

All views, opinions and statements are my own.