The Product Manager and Prioritization: How to evaluate a project’s objectives?

In almost every team, there comes a time when the Product Manager needs to evaluate and establish a project’s priorities. Experienced PMs know that you can’t just play “pin the tail on the donkey” with task priorities and that he/she will eventually have to skillfully select the most critical tasks out of the ones that have piled up. While performing research on the topic, I’ve discovered that both Russian and international companies split their prioritization procedures into two categories, or stages: the quick assessment and the slow assessment.

Quick & Slow assessment

First, the quick assessment is performed, which cuts off any irrelevant tasks. After that’s done, the PM conducts a slow and detailed assessment of the remaining tasks.

Quick assessment

Poker Planning

For the quick assessment, the PM ‘attacks’ the tasks with the Agile approach, which establishes and compares how resource-intensive and how useful each task or feature is.

1. First, the product manager and technical experts discuss how useful each new feature (task) is. The team votes with a simultaneous show of 1–3 raised fingers and the PM records the average value. This is repeated with every task/feature.
2. The next step is to discuss how difficult it is to implement the new feature (how hard it is to actually perform the task). The procedure for evaluating difficulty (cost) performed in the same way as with usefulness, by a show of fingers, and the voting results are recorded in the column “Average Cost Estimation”.
3. Then, the PM correlates the usefulness and costs for each task (feature). In the example above, the table shows that Feature 2 and Feature 3 are the strongest leaders, meaning that these two updates should be developed and implemented first. From the second column, we can determine that they are easy to make, and from the first, that these features will be useful to users.

If the assessment shows that the feature doesn’t have much value for users, but the product knows that it is important for the company, the feature can be included in the subsequent update, or can be taken out of the quick assessment entirely as a ‘must-implement’ exception.

Global Product Management community

RICE score method

There are plenty of options for evaluating priorities. For example, the company Intercom uses the RICE score method

• the first column contains the names of the features;
• the second column (Reach) — the number of people (or a score that represents it) that may encounter or use the feature;
• the third column (Impact) — a level of confidence that the update will be useful to users;
• the fourth column (Confidence) — the probability that the feature will be a “hit”;
• fifth column (Effort) — costs and complexity.

After these parameters are established, the RICE score is calculated: Reach, Impact and Confidence are multiplied and the result is divided by Effort.

In this case, Impact is established either by voting within the teams, or by user feedback.

Welcome to join — http://productstar.org/

Hierarchy of Metrics

Another option for performing a quick assessment is the “Hierarchy of Metrics”. Let’s take the social media company VKontakte as an example:

1. The top-level (Main) metric considers how often users interact with the feature(s).
2. The next level down shows the Service Metrics such as “video viewing time”. The PM can discuss what affects the video viewing time with the team: the duration and number of views. These are lvl 1 metrics.
3. At lvl 2, the metrics establish what specifically affects each lvl 1 metric. For instance, the duration is affected by the percentage of completed views and the duration of the video itself. Meanwhile, the view percentage is affected by the quality of the content and video download speed.
4. After each item is deconstructed into levels: we get a tree that stems from the main metric as the “trunk” and the levels that affect it as the “branches”. At this point, the PM can think about how the performance of these metrics could be improved (for example, by adding HD format to videos). But before you implement the changes, you need to understand what metric this will affect. Building on the example, adding an HD format will affect the quality of the video. The PM can examine the tree and finds the level where Video Quality is affected.

The closer the upgrade feature is to the Main metric, the more useful it will be overall. If the feature is at a very deep level, then there is a very low probability that it will affect the Main metric.

In this example, in order to measure how important the quality of the content is to the overall product, the PM needs to builds a tree that is then confirmed by analysts. After this is done, the developers will have tasks for six months to a year in advance. They will gradually modify and improve the product by completing these tasks.

Hierarchy of metrics in Excel format

Pictured above is the “Hierarchy of metrics” in Excel format, which can be used for monthly or quarterly analysis and prioritization:

• the first column shows the metrics that need to be improved with the help of new features;
• the second column shows the expected values;
• the third column shows new features that need to be added;
• the fourth column shows the importance of the features (on a scale of one to three);

What the PM encounters when working with the Hierarchy tree:

• In many cases, there can be 15 features that are included in the table, while the remaining 180 are waiting for their turn to make it into the hierarchy;
• When the tree is compiled, ideas for features can be generated not only from the table but from the backlog as well, due to the fact that team members can suggest something important;
• “Importance” is determined by comparing the new feature/service with past projects and features;
• If one project affects two or more metrics at once, it should be placed only under the metric on which it will have the most influence.

Slow assessment

To bring up an interesting feature, in Skyeng, there is a sort of “quest” for students, where they need to complete various tasks over the course of several classes. The quest begins a couple of classes before an upcoming payment for next month’s lessons, and in order to complete the quest, the user has to pay for the next month of training on the platform.

So we need to find out how much money this quest feature will bring and how to prioritize this task in relation to other features

To do this, we compile a calculator with answers to primary questions (the points in the table, where the PM is unsure of the accuracy of the data, are highlighted in yellow):

• How many users will encounter this decision point over the span of 12 months? For example, let’s say 100 000 people.
• How many users will try to do the quest at all? The team suggests that this could be 44%.
• How many users will complete the entire quest? Perhaps 81%.
• How many more users will make an upcoming payment after the quest feature is implemented? Presumably, consecutive payments will increase by 4%.
• How much profit do consecutive payments bring? The team knows for sure that each one brings $100 in profit.
• What added profit will the company make in 12 months from the launch date? We count the number of quest participants and multiply by the consecutive payment figure.

Global Product Management weekly online Meetup & AMA

Why are the numbers that are highlighted in yellow those specific values: why 40%, and not 60% or 15%?

In order to take into account the “unknown” aspect of these values, we calculate various probable scenarios for the unknown coefficients (point 2 in the picture), as follows:

1. Consider the pessimistic, realistic and optimistic scenarios for the number of people who try WOW-lessons (quests). Determine a value for each scenario that meets the corresponding expectations.
2. In the “Scenario Probability” column, determine a value for the probability that the scenarios will actually occur and calculate the average value. The same is done with the rest of the ‘unknown’ metrics.
3. Estimate the cost of development.
4. Calculate the ratio between the development cost and the estimated profits (for the development cost, the PM needs to clarify with the team members how many hours they plan on spending on development and then the results are multiplied by their hourly pay).

After the feature is released, the forecast for the large updates needs to be compared to the real profit data for the month.

To some people on the team, it may seem that the PM is spreading him/herself too thin and that prioritization is a waste of time. To dispel these worries (even if it’s just for yourself or for top management), it will help to calculate how much the company will incur in losses if the PM makes an error in prioritization. The result can often be an incredible amount of money.

A brief summary of the prioritization algorithm:

1. Establish the TOP-3 key metrics for a particular service.
2. Collect hypotheses for boosting these metrics: either from the backlog or somewhere else.
3. If the market for the product is new — use qualitative methods: ask potential users what they are currently using to meet their needs.
4. Perform a quick assessment and discard the “weak” or unnecessary features.
5. Perform a detailed assessment of the remaining features.

Skyeng uses one fast and one slow method for its prioritization needs and sets up the schedule for these procedures in advance. For example, the developers have a weekly planning meeting where the team discusses how much time each person will need to solve various problems that week. They provide the PM with their deadline estimates, after which the PM decides whether to go through with the development of that particular feature or not (backlog or elimination). When prioritizing the latter, Skyeng uses ROI metrics to show the ratio of the money that the company will make, compared with the money that will be spent on development. Sometimes the ROI can be more than 1000–3000%, while the development of the feature(s) will take a relatively small amount of time.

I wish you the best of luck in your competent prioritization strategy!

This post has been published on www.productschool.com communities

✅

If you’ve enjoyed reading the article or found the information useful, please:
1) subscribe to my channel (Click “Follow” to receive information about my next articles when they come out)😎
2) send a link to the article to one of your colleagues (or to your team’s chat) to spread the useful information further 🚀