How to know, what to work on next?
Setting priorities as a Product Owner on a live product
In 2016, I got the opportunity to become a Product Lead (= a Product Manager/Owner mix) at one of our great game teams at Wooga. Game teams at Wooga are independent product units. Each team has a couple of engineers, artists, game designers, product managers and a producer.
Game teams are embedded within a studio. Each studio has a different focus. In our case, that’s casual puzzle games. Besides the product teams, there are plenty of service teams coming up with great ideas you would definitely want to hear about and collaborate on. But in the end, it’s always down to us to make the final call on what to work on next.
Earlier this year, I decided to take a step back and summarise my thoughts on what it means to be a product owner on a mobile app with a large audience. Why? As with most jobs nowadays, I assume that many POs out there have entered their jobs with a lateral entry. So I hope, that my thoughts on this topic will be useful for new beginners and veterans alike.
In this text I am going to write about:
- The importance of knowing the costs and gains of your decisions and how to predict those
- Present a framework that helps you to prioritize new feature ideas
- Another framework, but this time for prioritizing iterations on existing features
What does a Product Owner do?
Simply put, as a Product Owner, your job is to decide what to work on next. To be able to do that, you need to understand the costs and the potential gains of each and every decision you make. Cost and gain as in actual money, but also in how it will impact your team, your long-term profitability and your overall user numbers.
There are many other (probably more elaborate) definitions of what it means to be a PO — especially for those in a typical SCRUM team setup. However, I found the definition above to be the most useful in my daily work life.
Costs & Gains
Defining product priorities often feels like looking into a crystal ball. Sometimes the outcome can be truly unpredictable. But in the end it always breaks down into this:
- Evaluate the technical complexity so you get a good idea of the cost factor.
- Evaluate the potential positive impact on your product, so you get a good idea of the potential gain.
These are the areas you will want to take a look at for evaluating a potential impact:
Monetization: Getting your users to pay you (with money, potatoes, love…it’s up to you really) or alternatively: get your users to save you money e.g. by reducing the costs for other departments.
Retention: Making your users come back more often / spending more time in your app.
Marketability: Marketing your product profitably (through paid and/or organic growth).
If you have a great team of programmers and a good understanding of your P&L sheet, then evaluating the cost factor will not be a problem. Your developers should be able to give you an estimate for how long a certain feature will take to build. Multiply that by the cost per day and add an additional 20% buffer. Easy.
But if you want to evaluate the potential positive impact (gains) on your most important KPIs, then things are usually much more complicated.
How to predict the impact on your product
If you are working on a live product, then creating something new is usually the most fun. There are likely fewer legacy issues and your users will notice new designs much faster which will make it easier to see spikes in your relevant KPIs. In summary, new is always exciting. But at the same time, you will also have less of an idea of what you can actually gain from your new features.
These three approaches have worked well for me in the past:
1) Exploit in-house knowledge
At Wooga, we are massively relying on A/B testing. Lucky for me, our Business Intelligence team is great at building internal knowledge sharing tools. As a result, each A/B test result is easily accessible via our internal reporting platform — in fact, the tool will not allow you to hide a result. It will always be there. Whether it had a positive or a negative impact on one of our products.
Most tests will not be overwhelmingly impactful. And while improving your revenue by 20% may be an extraordinary result that is totally worth sharing, your failures are just as important to your peers.
If your company does not have that kind of culture, or you don’t have access to such a knowledge sharing platform, then reach out to experts in your network. The least they can do is to make a good guess. If you ask five people in your company for an estimate and you use the average of these guesses as your potential positive impact on your product…chances are you will be relatively close to the final result.
2) Reverse engineer spikes in revenue, downloads or popularity
In nearly every market, you have competitors. That’s great for business and observing your competitors should be a key activity of your everyday job.
If you work on a mobile app, you probably know AppAnnie.com or one of their competitors. If not…you should check them out. These tools give you access to the download and revenue of a mobile app over time. All you need to do is to search for spikes and compare these with the release notes or changes in the design you noticed yourself.
In the screenshot above, you can see how an update around the end of July improved the revenue baseline for this particular app. It probably makes sense to check the release notes for that update. Whatever they did, they must have done something right.
3) Create multiple scenarios with what you know and fill in the blanks
If you don’t have all the necessary information, then you can still create business cases by relying on the data that you own with certainty. Let me give you an example:
You are working on a new feature that is supposed to increase the share of daily active users (%) to use feature X at any given day. Each time a user interacts with the feature you make $0.10 USD. The average user who engages with that particular feature uses it twice per day.
The question is: How much additional revenue can you expect from your new design? In the example above, what you don’t know, is by how much you can increase the share of daily active users engaging with feature X. So the best you can do is to create an optimistic and a pessimistic scenario.
An optimistic scenario could look like this:
10% (% of daily active users engaging with the feature) * Daily Active Users * $0.10 USD * 2
And a pessimistic scenario could look like this:
5% (% of daily active users engaging with the feature) * Daily Active Users * $0.10 USD * 2
Now you have a rough idea of what the best and the worst outcome of your new feature idea could be. Still, want to work on it?
If you have trouble with coming up with a good guess, just ask five people in your team to give an estimate, then take the average of these five estimates and use that.
Mapping out what to work on next
Now you know the costs and gains of your ideas. Great! But what to work on next? There are plenty of variables which can affect your decision-making process. And some of them can be impossible to control. Like suddenly deprecating APIs, SDK updates, a new OS release or simply a flu epidemic in your team.
In this case, let’s assume you have no deprecating APIs to take care of and all your team members are healthy.
The Slope of Inexcusable Risk & Missed Chances
Look at the chart above. The x-axis represents the cost factor of your new feature ideas, represented by time and/or cost such as an investment into a special technology. The y-axis represents the potential impact on your KPIs split by monetization, retention, and marketability. In this case I gave it a range of +0% to +20%.
Literally, all you need to do now is to take your feature ideas and map them out on this chart.
Done? Great. Next, you draw a line like in the chart below. I call this the slope of inexcusable risks and missed chances.
Everything above the line is a YES! And everything below that line is a definitive NO! This is a simple way to communicate the reasoning behind your decision-making process to your team and other stakeholders.
Working on existing features
Ok. We talked about new features. Great. But sometimes you really shouldn’t work on new stuff and focus on what you are already got. Chances are, not every feature in your product works that well. Maybe you had a feature that once brought in all the money but then started to work less well because of some changes you made sometime later. The good news is: What is broken can be fixed and what is working can still be improved. So it’s important to be aware of how well your features are doing over time and how many of your users actually engage with them.
What you want to know before working on existing features:
a) What % of your users is engaging with a feature on a daily basis?
b) What is that features impact on retention, monetization or marketability?
If you know the answer to these two questions, you can map out every feature in your product on the 4-field portfolio below:
The lines between these four areas come in many shades of gray. But they help you figure out what the core problem of a feature is and what you need to do, to make a change for the better. The image below is a generic example of how that could look like:
As you can see from the x- and y-axis, I am focussing on a monetization feature here. You can use actual numeric values or a more general description like the one you can see here. That way you can keep it more casual, which can be helpful if you discuss the results in a larger group setting.
In this example, I would avoid working on Feature 2, 3 and also 5. Instead, I would try and move Feature 6 into the top right corner (it’s nearly there…maybe there is something that can still be tweaked).
Have fun trying this out for yourself!
Thanks for reading!
I hope this was useful for you. Got anything to add? Hit me up on Twitter at @phlppstlzr or directly here on Medium.
