ORGANIZATIONAL ASSESSMENT

How to build a Good Product Team

12 questions to check the pulse of your product organization

Over the last four years as a Product Manager and Founder, I’ve often stumbled upon questions such as: What makes product teams successful? What processes do we need to put in place to ensure that we build a good product? And how can we measure the quality and performance of our product management team? The most obvious answer is:

Great product teams build great products.

In this article, I attempt to dissect this truism and tackle the question of what it takes to build a good product. The result is an actionable checklist you can use to assess the health of your product organization.

A checklist for assessing your product organization

1. Vision: Have you articulated your product vision?
2. Strategy: Do you have a clear product strategy in place?
3. Customer interaction: Are product managers (and engineers) speaking to your customers regularly?
4. Discovery: Do the people working on the customer and product discovery have the authority to influence your strategy?
5. Focus & Alignment: Have you established an effective and transparent prioritization mechanism?
6. Prioritization: Do you have a roadmap that prioritizes the most important business items?
7. Ownership: Does the product team own business outcomes (not just output)?
8. Outcome: Have you defined what success means for your product?
9. Traction: Are you measuring the success of your product?
10. Progress: Do you have the processes in place to continuously transmit progress and the traction of your products to the organisation?
11. Delivery: Can you ship the thing you intended to ship, in the timeframe that you had planned?
12. Feedback: Do you have a process in place to continuously transmit customer feedback to the product team?

Let’s dive deeper into each question, discuss why they are important and explore actions as well as examples of how product managers and leaders can improve in each dimension.

For context: At store2be, we’re building a platform where brands and live marketing agencies can book promotional spaces in physical locations such as shopping malls, train stations, airports, public spaces and more. Our platform team consists of seven engineers, two product managers, a product designer, CTO and CPO.

1. Articulate your product vision

“The product vision describes the future we are trying to create, typically somewhere between 2 and 5 years out.”

Marty Cagan · Vision vs. Strategy

Why is a product vision important?

Articulating a clear product vision is the foundation for where your company is headed and provides long-term guidance of what the company wants to achieve. It serves as a North Star for the team. It’s very difficult to prioritize initiatives if a common understanding of where you want to be is not shared among the team.

At the same time, I would encourage Product Managers to leave some flexibility and space to iterate on your product vision during the early stages to incorporate customer and market insights. I’m sometimes surprised by how top-down all the existing frameworks ‘dictate’ strategy and roadmap based on the vision statement. Vision and strategy should leave some space for integrating Product Discovery insights. When we started out, our vision was to build the “Airbnb for retail space”. The v1 of our platform turned out to be an altered version of Airbnb rather than a platform that solved a market need.

How can I create my own product vision statement?

Geoffrey Moore’s Product Vision Statement is a good framework to start with. It brings together for whom the product exists, the problem/opportunity, the core benefit, and the differentiating features in a concise format. Alternatively, the Lean Canvas is a more detailed, established business modeling tool that helps outline your vision.

Source: Geoffrey Moore / via Prodpad

Examples

Great vision statements focus on the user needs and long-term perspective.

• Stripe is a technology company that builds economic infrastructure for the internet. Businesses of every size — from new startups to public companies — use our software to accept payments and manage their businesses online.
• Netflix: Becoming the best global entertainment distribution service. Licensing entertainment content around the world. Creating markets that are accessible to filmmakers. Helping content creators around the world to find a global audience.
• store2be: Our mission is to enable brands to interact with people in the real world. We are building a global platform where brands can book scalable, targeted and measurable Live Marketing campaigns.

2. Put a clear product strategy in place

Source: Roman Pichler

Why is it important to have a good product strategy?

1. “A bad plan is better than no plan.” — Peter Thiel
2. It connects the dots between where you want to go and where you are.

How can I put a good product strategy in place?

Marty Cagan defines product strategy as “our sequence of products we plan to deliver on the path to realizing the vision.” Product strategy is often compared to the roadmap, but there’s more to it. And while the question of how to define a product strategy and existing strategy frameworks deserve a piece on its own, I believe a good strategy covers the following aspects:

• Target customer(s)
• Customer needs and Jobs-To-Be-Done
• Key differentiators / Positioning against competing solutions
• How you are going to build your key differentiators
• Pricing and Business Model
• The order in which you build solutions (‘features’) to perform those jobs for each specific customer segment

Expanding on the last point, your early-stage strategy should answer the question “what hypothesis do we need to prove first, second, third? And what are pre-requisites to achieve that?”. Asking those questions forces you to come up with a reasonable approach to accomplish your value proposition. An example where this can be effective are marketplaces that are facing the chicken-and-egg problem.

I’m planning to expand on strategy frameworks and how a strategy is formed is a topic for another article.

Communicating your strategy

Finally, ask your entire team the question: Can you say what our strategy is? Communicating (changes to) your strategy regularly and aligning all departments around it is as important as defining one. You have a clear strategy in place only once every team member has understood it and is able to define their objectives based on it.

Example · The strategy of store2be

At store2be, we decided to pursue a three-step strategy to establish a platform for Live Marketing campaigns. Each stage addressed certain needs we observed in the existing marketplace, aimed to solve the chicken-and-egg problem.

1. Build a SaaS tool for retailers: At first, we built and sold administration software to retail space operators to digitize their assets (spaces). This helped us acquire inventory for our marketplace.
2. Provide brands with offline marketing analytics: During the second phase, we built an analytics solution for advertisers to measure the performance of Live Marketing campaigns. We differentiated our services for advertisers from existing space brokers and location owners.
3. Build and scale a booking platform: With the underlying software infrastructure in place and a key differentiator for the demand-side of the market, we are now establishing a SaaS-enabled marketplace for booking retail spaces.

3. Speak to customers regularly

Why is regular customer interaction important?

Good Product Teams develop a deep understanding of their customer and their needs. They do so primarily through regular, direct customer interaction, effective research methodologies, and data analysis. ‘Direct’ is key here because communication through other parties, such as sales and customer support, can be filtered or biased and you miss out on the opportunity to conduct proper research. ‘Regular’ is important because your product changes, your customer segments change and you otherwise risk relying too much on information gathered in the past.

How can I establish regular customer interaction?

Marty Cagan recommends a minimum of three 1-hour customer interactions each week ongoing.

1. Ask yourself: How often are you and your team speaking directly to customers?
2. Define your desired cadence and add it to your checklist
3. Get on the phone and call your (potential) customers. From my experience, the majority of people love talking about their work and the problems they encounter therein.
4. Involve engineers and other stakeholders regularly in customer interviews and user tests. It’s an opportunity to reduce communication/alignment overhead and spark more joy for solving user problems.

Customer obsession

“Good product teams are customer obsessed.”

Good Product Team/Bad Product Team · Justin Bauer

I personally like the term customer obsession used by Amplitude’s VP of Product Justin Bauer. Customer obsession is difficult to measure, but a good start is to anchor the customer needs in your vision and objectives. You can also observe customer obsession: How excited is the team about interacting with customers? How attentive are they to customer problems? How does the team react to customers getting stuck in the product? Product Managers play a key role in establishing this mindset in a company.

4. Give people working on customer and product discovery the authority to influence strategy

Why is this important?

Inspired by the book Product Leadership, this question addresses any potential disconnect between company management and your product discovery and delivery teams. In an era where tech companies have proven the potential benefit of autonomous, objective-driven teams over top-down company hierarchies where feature roadmaps and deadlines are handed down by management, it’s vital that those doing the customer research have the authority to influence the company and product strategy.

How can I establish this?

Ideally, strategizing is an ongoing, joint effort between product teams and management. The strategy should primarily be driven by the teams generating market and customer insights rather than based on assumptions made by management in brainstorming meetings. Don’t miss out on the opportunity to empower cross-functional teams to strategize and plan on their own, then incorporate this in the company strategy.

Example · Authority of Product Managers at store2be

At store2be, we are moving towards a culture where Product Managers have full ownership over the product strategy within their respective fields. It is the PMs who primarily drive the roadmap and align it directly with executives and other stakeholders. They seek to understand and incorporate the perspectives of other functions. This has the added benefit of Product Managers becoming more authoritative figures in the company.

5. Establish an effective and transparent prioritization mechanism

Why is this important?

A clear prioritization mechanism removes opinionated discussions and enables effective roadmapping. It also enables product teams to maintain focus on building a ‘deep product’, as they can confidently reject ideas and initiatives that do not fit the current strategic focus.

How can I establish this? Examples of prioritization techniques

As a novice or seasoned Product Manager, you’ve probably studied some or all of the following techniques:

• Value vs. Complexity
• RICE
• Kano Model
• MoSCoW
• Weighted Scoring
• …

After trying several of these mechanisms I noticed they can all be effective depending on your situation, however they do not incorporate strategic focus very well. If you prioritize 200 backlog items, you risk ending up with a bunch of disconnected themes across your product portfolio that all seem important. I would therefore apply another prioritization layer more than anything: strategic themes.

Example · Strategic focus over prioritizing backlog items

While it’s essential to consider cost and opportunity when prioritizing, I would argue that having a clear objective- and theme-based prioritization is even more important than a ‘low-level’ mechanism such as weighted feature scoring. Theme-based prioritization can happen through Story Mapping, based on Customer Jobs-To-Be-Done or a specific North Star Metric. Set the strategic theme, and if necessary you can then sort and rank stories according to one of the ‘classical’ mechanisms into these themes. As a side benefit, this cuts hours of guesstimation work on stories that don’t align with your strategy in the first place and frees up time to properly validate a few core opportunities before building them.

6. Create a roadmap that prioritizes the most important business items

Why do I need a roadmap?

The first purpose is because the management of the company wants to make sure that the teams are working on the highest business value items first.

The second purpose is because since they’re trying to run a business, there are cases where they actually need to make date-based commitments, and the roadmap is where they see and track those commitments.

The Alternative to Roadmaps · Marty Cagan

This statement resonated with me because:

• A roadmap serves as a forcing function for prioritization.
• It creates strategic alignment in the company.
• It provides some planning ground.
• It’s a great communication tool.

We’ve always had to balance our product roadmap with external deadlines, for example when onboarding strategically relevant suppliers into our platform.

How should I build my roadmap?

Roadmap formats are a heavily discussed topic in Product Management. In principle, the product roadmap should define actionable, shared, high-level goals for the product team. Those goals should describe desired outcomes rather than outputs (=features). An effective method for this is to prioritize the most important customer needs and business outcomes and giving the product team full authority and accountability from there. It is then the Product Manager’s job to discover a roadmap that can deliver on these targets.

Based on their discovery and validation work, this allows product teams to come up with an effective, more short-term roadmap of initiatives. Finally, ensure that your roadmap is up to date and visible to the entire team to serve as a central and educational reference for the entire company.

While there is a lot of discussion around roadmaps, I still believe that they are one of the most important communication tools that can establish a shared understanding of what the roadmap contains and means for your company.

7. Establish ownership for business outcomes (not just output) in the product team

Why is this important?

This question addresses the desirability of autonomy, authority, and accountability of a product team. When Product Management teams are ‘handed down’ a feature-based roadmap, they are forced to take on the role of project and delivery managers, which undermines the potential of more autonomously product squads.

How can I establish ownership for business outcomes?

In the spirit of what I call ‘modern product management’, product teams should be responsible for business outcomes and able to move freely within the opportunity and solution space. I recently stumbled upon the below illustration of a Continuous Discovery by Teresa Torres. It illustrates the ‘negotiation space’ between Product Leader and Product Team. At an early stage, outcomes might be negotiated in shorter intervals. On a side note, a quarterly rhythm also aligns well with company-wide OKRs and is a good time horizon to reflect on the previous quarter’s hypothesis and experiments. This is a topic I’m planning to expand on in future articles.

Example

Source: PRODUCT TALK

8. Define what success means for your product

Why is this important?

The most obvious answer is that great product teams build great products. But if you can’t measure it, you can’t improve it. You need a definition of what success means at each stage of your product.

Success of your product is where user value and business objectives come together. Let’s express this in the form of two question sets:

• What does success of your product mean from a user POV? What desired result should the product enable the user to achieve? For Superhuman, for example, success might mean staying on top of my emails, improving my response times, having ease of mind or similar.
• How is product success reflected in your business objectives? Increase Revenue, sign up rates, transaction frequency, matching quality?

User success > business success

When you think about it, it’s surprising that Product Teams often measure their success in terms of business metrics, rather than user success. I’ve made this mistake many times. I believe it happens because every single driver tree of a business metric, such as revenue or profit, will naturally result in objectives that aim to move the underlying metrics. This can create wrong incentives for product teams that don’t focus on the user problem, lead to bad UI/UX and delay the process of reaching product/market fit.

Resist the pressure of optimising conversion rates and financials at the cost of UX, especially before you have reached product-market fit.

A good product solves a user problem in a way no competing solution does. When you’re setting out to solve a user problem, anchoring customer success in your core metrics is an effective way to get you there. Most of the products I like to use put the user experience first. If you look at the products most popular with end-users — Asana, Superhuman, Slack — they all spent years optimizing the user experience before optimizing for business metrics.

How can I implement this?

Being clear on what jobs your users are supposed to be able to do with your product is an important first step. Once this is done, you might ask your users how satisfied they are with your solution in performing a given Job-To-Be-Done. How does it benchmark against competing solutions?

In a recent conversation with a friend and early-stage founder working on a software tool for business customers, I asked how they prioritize their product initiatives. He explained to me that they are optimizing for “flow” — meaning how long the user can engage with the product as part of that specific workflow, completing their job, without having to switch the tool. Similar to working front to back through a customer journey, this gives the team a clear focus to prioritize for. I think this approach of optimizing for ‘User Flow’ is a great way of putting the user first, which eventually maximizes the value of the product and business.

9. Implement processes to measure the success of your product

TL;DR

• Keep your analytics setup lean and focused on your core metrics
• Include qualitative insights and don’t limit yourself to quantitative metrics
• Differentiate between usage and performance/satisfaction of your product
• Segment your users to understand product/user segment fit

Examples · Choose your metrics

In reality, product teams need to balance a set of qualitative and quantitative objectives of your product, both on the user and the business side. Some Examples:

• User surveys: Product/market fit score, NPS, Jobs-To-Be-Done satisfaction*, qualitative customer feedback
• Product metrics: Signup rates, conversion rates, usage frequency, retention, transaction frequency, revenue

*For example, you may ask users with a Likert Scale: How satisfied are you with the way you can do X with solution Y?

Set up lean analytics

When I launched my first websites, I would usually just plug in Google Analytics to have any kind of analytics setup. It’s good to start tracking from the get go as you might need the data later, even if you don’t look at it in the beginning.

That being said, avoid overanalysing. Don’t overrate no. of website visitors and signup conversions in the beginning as it might distract you. Chose a core set of metrics that measure what you defined as success of your product. Ideally, focus on usage and retention metrics for a key user segment. Then add monitoring for a bunch of additional metrics that you want to have on your radar. There are heaps of great analytics tools, such as Mixpanel, Amplitude, Hotjar, etc. which are not in the scope of this article.

Include qualitative insights

Another mistake I’ve made in the past is to force a quantitative measure upon ourselves before even acquiring the first recurring user. A conversion rate might not be relevant if you have a small user base of B2B customers. This will help you focus on building a great product first before optimizing conversion funnels. No product metric can replace the insights you will get from speaking to 20 users. Incorporate qualitative feedback into your regular reporting.

If the user base is too broad to give you a clear view, segmenting into cohorts might help you as well. We spoke about product strategy being a set of features we build for a certain target group — maintain focus on this target group.

10. Continuously transmit your progress and the traction of your products to the organization

Why is this important?

Teams need to have the processes in place to continuously transmit their progress and the traction of their products to the organization.

Whether it’s reporting the metrics of a live product, the status of one in development, or the insights from user testing — a lightweight way to project the status of metrics the company cares about, available to anyone who wants to see them, creates a virtuous cycle in a product group.

7 heuristics for being a product director · Brandon Chu

I wholeheartedly agree that the output of the team and the outcome the product needs to be visible and communicated regularly to everyone in the company, for many obvious and some non-obvious reasons.

How can I implement this?

Over the years, I’ve found a few mechanisms effective in transmitting progress to the organization:

• Product Newsletter: In the beginning, we would write a monthly newsletter to the entire team describing new features and what is planned next
• Product Crunch: A bimonthly all-hands meeting with a recap of the product strategy, a demo of new features, usage metrics and things we’d work on next
• User-focused Release Notes: Our Product Managers now write release notes with screencasts demoing new functionality in our platform. Our sales team, for example, gets informed about updates and onboarded into the product at the same time. Make sure those notes convey the user value, not only the technical details.
• KPI Dashboards: We have long anchored product metrics in our OKRs and thus made them visible to the team. We would make live dashboards visible to the entire team more in the future, showing indicators for product success (or failure).

11. Establish a development workflow to ship the thing you intended to ship in the planned timeframe

Why is this important?

This question addresses the health of your delivery process. Good product teams ship.

How can I establish this? · Indicators of a healthy delivery process

Good product development teams release regularly and develop a good sense of what work they can get done within a development cycle. I therefore suggest looking at two factors to measure your ‘delivery pulse’.

1. Release cycle
2. Ability to produce desired results within the release cycle

Note this is not about estimating the number of hours for a task. It is about managing focus, optionality, and constraints within a development cycle correctly to make sure the core functionality makes it to the user in the intended time frame.

Example · How we did it at store2be

Our Platform Team almost doubled in 2018, and we didn’t adjust our ‘Scrumban’ way of working. Development slowed down. Large epics would drag on sprint after sprint. We weren’t able to predict when things would be done. Epics suddenly took months to make it into production.

At the beginning of 2019, we adopted the core principles of Basecamp’s ShapeUp product development methodology. Our Product Manager David, who spearheaded this change, has written this great article on our implementation of ShapeUp, its benefits and things we’re still figuring out. You can also listen to this Changelog Podcast with Basecamp’s Head of Strategy Ryan on Basecamp’s approach to product development.

12. Put processes in place to continuously transmit customer feedback to the product team

Why is this important?

Beyond customer research done by the product team itself, there are plenty of other channels that generate valuable customer insights: marketing, sales, customer support. You need to establish a mechanism and mindset to ensure this feedback reaches the Product Team.

How can I establish this?

• Reduce the friction to forward feedback. Make sure there’s an easy tool where everyone can leave their feedback asynchronously. Measure the feedback pulse and celebrate it.
• Be a broken record about it. Make sure everyone knows the importance of customer feedback to you as a driver for product success.
• Finally, establish trust. I feel like the effort other teams put into forwarding and summarising customer feedback to Product Managers is a measure of their trust towards the Product Team to make use of it effectively.

Conclusion

Scope

While I believe many of the above thoughts can be applied to Product Teams in general, my personal experiences come from an early-stage SaaS and platform product. I’m eager to hear your thoughts on what an assessment might look like for different company sizes and product stages.

How to use this

You can start out by doing this assessment together with your team. Create a survey and let your team assess: In which areas do you already feel strong? Where do you see gaps? Spark a discussion and see what improvements you come up with. I set up a recurring health check to reflect on some of these questions regularly with the team.

Diving deeper

Each of these questions probably deserves more than one dedicated article on their own and I am planning to expand on some of these topics in the future. I’m also thinking of transforming this into an actual assessment spreadsheet and categorize questions into vision & strategy, defining and measuring product success, delivery, communication, etc.

Share your thoughts

I hope this checklist and my thoughts were valuable to you in assessing your product organisation. There are probably several more items that should be added to the checklist. Some of the items on the list might not be applicable to you. Either way, I’d be curious to hear your thoughts, get feedback and learn how other product leaders are assessing and improving their product orgs! Join the discussion below. 👇

Thanks to David Arens, Jan König, Arne Zeising for reviewing drafts of this article.