Product vs. Project Managers: Marty Cagan’s Twelve Best Lessons for Product Team Work
“Product is hard.” Marty Cagan says to an audience of eager product people at Appcues in Boston, Massachusetts, and Pivotal Labs in New York City. “And for product managers who are trying to help companies transition from a traditional project delivery organization to a modern product culture; When working in certain types of organizations (like traditional banks or insurance companies), there are a lot of ways that a product team’s purpose can be misunderstood and work can go wrong.”
Marty is a household name in the product community, as he travels around the country and world, helping those who are trying to navigate their respective workplaces by discussing why he believes true product management techniques can win in today’s companies.
He also believes that when product and project leaders learn to distinguish between the roles and objectives of product management and project delivery teams, they will learn how to increase their effectiveness and become successful working together. Here are the twelve best product management career lessons I learned from Marty.
1. Understand the Agile Method
Recently there has been a lot of backlash against techniques such as agile, lean startup and design thinking, according to Marty. He reassures that there is absolutely nothing wrong with these methods, for they can be very helpful when used in the right ways. However, when people use agile techniques incorrectly, confusion arises. As he states, “There are fundamental things agile development strives to accomplish, and those are the only things that agile techniques work best for.”
Most of the benefits that product and project teams get from using agile techniques is learning to do product releases at least every two weeks (like at Zappos), as opposed to every three months. Some product teams (like at Amazon or Etsy) even release once or many times a day.
“We need consistent, frequent release vehicles.”
But to get there, Marty explains that product teams are expected to undergo a level of ‘test and release automation’ as he calls it, and that changes all of the company’s dynamics. The changes are never easy, but necessary.
2. Function Like a Lean Startup
One of the ideas behind lean startup techniques is needing to learn fast in order to innovate. To innovate, one must perform many iterative tests, and opportunities for iteration is limited at startups. Most startups only have around 12–24 months before their seed funding runs out, and larger information technology organizations will vary in the number of attempts they get within a 12-month time frame. The number of opportunities teams at larger companies get largely depends upon the amount of risk project leaders are willing to accept.
“Innovation is function of iteration. It is a function of trying out ideas. And we need to learn fast in order to innovate.”
While a typical startup may only get three to four attempts at iteration within four to six months for example, Marty believes that in order to solve most worthy problems, startups should be pushing out 50–100 attempts.
“I tell founders that if you want to have any hope of solving problems and build the right products, you’re going to need to have at least 50 to 100 iteration attempts. And there is just no way to do that by using your engineers to build four-month MVPs. If you have your engineers build a four-month MVP and deploy it, and it doesn’t actually do what you want, you have just wasted four months out of an 18 month runway.”
3. Project-Led vs Product-Led Companies
Many organizations today view their information technology teams as servants to the business. In organizations with a product mindset, teams are there to serve customers in ways that meet the needs of the business. It’s a tough pill to swallow, but Marty admits that the reality is, a true product manager role has more relevance in organizations striving to be product companies.
What tends to happen at organizations that claim to use agile, but with a project-led mindset — they neglect to empower their product teams. As Marty explains, “They seem to forget that the foundation of the agile manifesto is to adopt a product mindset and allow teams to own the problem solving.”
“If your company doesn’t employ agile in a way that empowers your teams, then you don’t need product managers; you need business analysts or project managers,” Marty says. And while he claims the product manager role evolved from the business analyst role, based on my own experience, the reasons for this is unclear. If anything, business analysts at large companies are given the ‘product manager’ title because it’s more fashionable.
As someone who has worked closely with business analysts and product managers for several years, I’ve learned the following distinctions:
Product managers specialize in learning about areas where problems exist for the business via its customers; and use their skills to generate potential solutions with the use of prototypes, experiments and iteration. Business analysts specialize in learning about business problems, and use their skills to document requirements for designers, engineers and stakeholders.
While these roles may appear to be similar at first glance, the differences are clear: product managers generate value via customer advocacy, prototypes, experiments and insights, while business analysts generate value via understanding the business, generating documentation and deliverables.
“Let teams take ownership of solving problems.”
4. Product Discovery vs Product Delivery
According to Marty, the two essential problems in building products are:
- What do we build (product discovery)
- How do we build it (product delivery)
“We need to realize that because these are essentially two different questions, and we need to approach them with two different strategies,” Marty recommends. “Many organizations today still don’t recognize product discovery and delivery as two activities that need to be separate. Many companies try to perform these activities as one, using the same tools.”
“Are we building the right product and are we building the product right?”
While speed to discovery and speed to implementation are important, conflating the two strategies ultimately creates avoidable amounts of technical and UX debt. Product and project teams need to understand the difference between discovering a solution and delivering a solution; because by doing so they can work together to formulate strategies that complement each other.
I’ve witnessed project teams use prototyping tools to share wireframes and prototypes with stakeholders, and in turn use the same deliverables to communicate to engineers what to implement. While I can understand why this is done, the approach is incomplete. Prototyping tools are created for product teams to quickly generate and communicate ideas for experimentation, learning, and refinement. Once the ideas are validated quickly and accordingly, project teams take the learnings and prototypes to create requirements, user stories, use cases, and concept models, which are then documented for designers, engineers, and stakeholders. While some may feel that creating requirements documentation is a waste, it’s only a waste if the documents are not useful. The trick is, create enough documentation that engineers can use, because it protects everyone from legal liability.
“Discovering a solution is different from delivering a solution.”
AirBnB has devised their own ways for distinguishing between discovery and delivery. According to Marty, AirBnB refers to discovery as ‘building products that don’t scale,’ and delivery as ‘building products that do scale.’ The reason for this is that early in their life as a startup, they spent too much initial time building products that would scale before they knew what was feasible and valuable. This caused the business to suffer a great deal and forced them to rethink their approach.
5. Validating Products vs Discovering Solutions
When discovering a solution, according to Marty, you are looking for something that is valuable, usable, feasible, and viable. In other words, the objectives are to build a product that is:
- Valuable: something that people want
- Usable: something people will understand how to use
- Feasible: something the business can afford to build, while keeping in mind that designers and engineers can build it with the time, technology and resources available
- Viable: The business has to be able to sell it and profit from it
Therefore, as a product manager working with a project team, the balancing act is to allocate time to discover what customers need, with allocating time to build what works for the business and customers.
“Prototypes are done in discovery. Products are done in delivery.”
Marty has noticed, in his experience, that teams at startups tend to build products that gravitate towards market validation, more so than truly innovative solutions (Apple is probably one of the exceptions, because they’ve built truly innovative products — like the personal computer, iPod, and iPhone). This is most likely because market validation tends to be more comfortable for startups, because you can get access to market feedback. However, the issue is rarely rooted in the market. This is evident especially when there are alternative products in the market that people are already using. As Marty explains:
“If your product fails, it isn’t because people don’t need their problem solved. It’s because your solution isn’t positioned to be better than the rest. In fact, in order to be successful, your product needs to be groundbreaking and be ten times better than the others.”
This is possibly the reason why the iPhone, Facebook and Instagram surpassed the competitors. Apple and Facebook figured out solutions that were not driven by market trends, but were driven by what customers liked about these products in the first place: they are simple, elegant, delightful, easy to understand, and just fun to use.
6. Minimum Viable Products vs Minimum Viable Prototypes
There are many different techniques and processes that can be used in discovery. One of Marty’s favorites is the design sprint. Similarly, there are lots of techniques and processes that can be used in delivery. “When doing discovery iterations, you aren’t building things that scale; you’re prototyping. Discovery is not delivery,” Marty says.
Marty believes that there are different kinds of prototypes that product teams need to be competent in. In his book “INSPIRED” he describes them as:
- User prototypes
- Feasibility prototypes
- Live-data prototypes
- Hybrid aka “Wizard of Oz” prototypes
Each kind of prototype is designed for specific objectives, which also comes with their own flaws and risks. And some prototypes are meant to be tested qualitatively, while others are meant to be tested quantitatively.
- Quantitative testing: Testing that tells us what’s happening or not
- Qualitative testing: Testing that tells us why things happen, and if there is a problem, we can test to figure out a solution
Prototyping in the discovery stage is important for the delivery stage, because engineers get the opportunity to optimize their work based what they learned from the prototypes, then deploy their work with confidence.
A designer who does prototyping can afford to create prototypes with flaws, because the purpose is to learn quickly. In other cases, engineers build prototypes as well, for specific reasons (e.g. building live-data prototypes to measure data management performance). However, if engineers deliver products with flaws into the market, it can hurt their reputation. Competent engineers will tell you, that when it comes to product delivery, the product needs to perform well, and be reliable, scalable, maintainable, and even globalized and internationalized if the requirements call for it.
Since discovery is not delivery, it’s a reason product managers and business analysts need to be clear about the purpose of releasing MVPs. Marty suggests a way to mitigate this by articulating what ‘MVP’ means in different contexts:
“The easiest way to fix the confusion when it comes to building a ‘Minimum Viable Product’ is if you just communicate in situations when you’re doing discovery that you’re building a ‘Minimum Viable Prototype,’ many problems are solved. Prototypes are done in discovery. Products are done in delivery.”
Minimum viable prototypes are made to see if there is a better way to achieve your design goals. In those moments, product managers find that design input valuable because it informs their product decisions. Minimum viable prototypes are also made because you want your engineers to be able to advise you on the direction of the product; because you need to determine if the minimum viable product is technically feasible. Business analysts find that feedback valuable because it informs the product requirements.
7. Product At Scale: Build Products You Can Sell
In the commercial or private sector, to ‘sell’ something means you’ve convinced customers to use your product for a price. In the public sector or within an organization, selling can mean that you’ve convinced stakeholders to use your product because it will increase productivity while minimizing friction and expense. Regardless of the meaning, building products that scale is about improving the quality of life for customers.
Product managers tend to think about the fastest and cheapest ways to test ideas, which among other things, drives their need to talk to customers in order to learn about the pain points. It’s within conversations and pain points product managers can discover the best ideas successfully.
Business analysts and project managers tend to think about the fastest and best ways to deliver solutions for the customer. It’s what drives their need to satisfy the business. It’s also in their ideas and deliverables they find success.
With these distinctions in mind, there’s a tag team opportunity. While product teams focus on the customer component, project teams focus on the business component. And when insights or tradeoffs are discovered that impact respective components, information is exchanged, agreements are made, prototypes are refined, and products ultimately are built, sold, and delivered.
Here’s a trick that Marty suggests can help sell your product at scale. Product and project teams can identify pilot users or ‘reference customers’ as Marty calls them to use the prototypes and products. If they like it, they will tell others, which leads to greater adoption.
8. Create Two Roadmaps: Feature-Driven & Outcome-Driven
One of the common anti-patterns that Marty often sees is companies still being addicted to feature-driven road maps. “Road maps are almost always the same thing; prioritized lists of features and projects,” He says. “The Microsoft Bing team recently confessed that only about ten percent of the road map ideas actually pan out. Unfortunately, this trend is pretty consistent across industries.”
I’ve grown to accept in my career that it’s impossible to stop project teams and stakeholders from thinking in terms of timelines and feature lists. So as a product person, it’s up to you to not only be the advocate for understanding situations before discovering solutions; you’re also responsible for raising the question, “What do we want to see happen as a result of adding this feature?” In other words, “What is the desired outcome?”
A way to address this challenge is to create a compromise, and create two roadmaps from two contexts:
- Feature-Driven: A list of desired content areas and features, arranged on a projected timeline
- Outcome-Driven: A list of desired business, customer or user activities and scenarios, also arranged on a projected timeline
Now, with two roadmaps we have another tag team strategy. Product teams focus on the outcome-driven component, project teams focus on the feature-driven component. And ensure that both teams have access to both roadmaps to ensure that at least the timelines are in sync. When insights, tradeoffs and redundancies are discovered, coordinate with each other accordingly.
9. There Are Two Kinds of A-B Testing
A-B testing is somewhat of a gold standard of techniques for product development. However, some people can get confused because there are actually two different kinds of A-B testing, as Marty explains:
“There is discovery A-B testing and delivery A-B testing. They are both the same concept, but they are used differently and for different reasons. Discovery A-B testing is much more limited and geared towards finding evidence. The bar for results is lowered, but the test is completed much faster. Some discovery A-B tests will even be invite-only, especially if it is B2B, rather than B2C. Delivery A-B testing, on the other hand, is usually done for statistical significance.”
Another issue that Marty often sees is when companies only optimize and run A-B tests, and then call that product management. Granted, every product company should be doing this, but it isn’t the only thing they should be doing. “Product managers are supposed to be creating value, not just optimizing and capturing it,” He says.
10. Managing Ethical Risks
Two great product companies, Ebay and Facebook, had very different approaches to ethical risks. When Marty worked at Ebay in its early days, he recalls the reputation system being one of the most important features to them.
“The biggest group of people working at Ebay was the Trust and Safety Team at that time, because they knew that for their customers, trust was everything.”
For Facebook, on the contrary, trust apparently was not everything. They didn’t approach their business with ethical risks in mind. And now, especially in the wake of the Cambridge Analytica situation, Facebook has been working passionately to mitigate these challenges.
While leaders are usually to blame for ethical issues, the product managers and designers are often the ones that realize these issues first. Leaders present goals to the team with little or no thought of ethical risks, and it’s when those goals play out that ethical issues arise.
When they arise, it’s important that product managers handle them correctly and effectively. Rather than just bringing up the issue to the leaders, they should be coming to them with solutions. Product managers could explore various ways to mitigate ethical risks by measuring the impact on customers, and business analysts could examine the market landscape to identify and support the management of risks on behalf of the business.
11. Competent Product Managers
Marty defines the competent product manager as having four things:
The first is a deep knowledge of the users and customers. This seems like a daunting task, but if a product manager just gets out of the office and talks to users and customers, they can easily acquire this deep knowledge.
The next thing a competent product manager needs to have is a deep knowledge of the data that customers generate. To achieve this, a product manager needs to utilize things like web analytics tools, sales analytics tools, and some form of data warehouse tool that shows how the data changes over time. “Most successful product managers begin their day with dedicated time with those tools so that they know how to answer questions that may come up throughout the day,” Marty says.
The next, and possibly most difficult, part to being a competent product manager is having a deep knowledge of the business. As Marty explains:
“A product manager needs to understand how their product is marketed, sold, paid for, and monetized. They need to understand the legal, privacy, partnership, and contractual constraints. The only other job in a company that requires this level of knowledge is CEO.”
Finally, a competent product manager needs to have a deep knowledge of the industry. They need to understand the competitive landscape, and perhaps most importantly, relevant market trends.
12. Product and Project Leaders Can Work Together
Based on everything Marty explained, I was once again reminded of the following points all product and project people need to understand:
- Project delivery teams determine the best ways to allocate resources to build and deliver products on time to stakeholders and users.
- Product management teams determine the best ways to build products while solving problems for stakeholders and users.
And all that remains is for project delivery and product teams to collaborate to ensure that they succeed in their roles, ultimately ensuring that products perform well in the market.
I hope that Marty Cagan’s twelve career lessons can bring you value, because at the end of the day, what product and project teams have in common: both are held accountable for product success.