What do PMs do?
Product managers are called the mini-CEO of product. In fact there are a bunch of definitions to define the role of product management. But in general, product managers are responsible for success and failure of their product. They plan for the features, they plan for the improvements, they plan for the growth, etc.
What is Testing?
The difference between a product and a project is the amount of ambiguity and uncertainty in the product. When running a project you know what is going to be delivered. The scope is well defined, the timeline is clear and you’ll deliver all the requirements at the end of a promised timeline. While in a product that you are developing for end users especially when it is a new product in the market and there isn’t many competitors or similar products in the market to take a look at what they have done, you are in an absolute uncertainty especially in the first steps. What do I mean by uncertainty? Depending on how your company thinks about the product your uncertainty level might differ. In some companies, it doesn’t matter what you build for the user (for their own reasons). They only care about delivering and developing the product. This means that end user is not in any parts of your picture. In other companies product managers listen to their users and try to create the best experience for users while solving their problems. This means that you don’t have an eye on what users might want or how they might react to your new change in the features.
This is when testing becomes important. You don’t know or you are not sure if users would like the new feature or change you want to implement to the product or not. So you need to test it before publishing it. We call it running an MVP version of your product. An MVP version is a very small version of the product you are thinking of which still delivers the value. Another reason for making an MVP is saving resources and time. Developing a whole product will take a lot of time from you and the team. Saying yes to a feature means saying no to 10 other features. So as a good product manager you need to be selective. One solution to being selective is creating MVP versions of your new products then implementing it. The test version also doesn’t need to be tested on all users. You can segment your users and run the test on them only. Here you also need to be selective when choosing the segment of users. Select those who are more related to the test.
Running a test isn’t just about creating an MVP and running it on a segment of users. It needs:
- Planning (You need to know what the main goal of the test is then try to break it into several phases if possible. Maybe you need to do the test in several steps and each next steps depends on the result of the previews step.)
- Scope design (Knowing the goal behind the feature, regardless of being a completely new feature or a part of an older feature, defines the scope of the test. Sometimes you can’t really break something to a few steps to test and it’s not testable — which I believe everything is testable- .)
- KPIs to be set (What should happen so that you call the test successful? What are the expectations considering the test being just an MVP?)
- Timeframe (Based on your understanding from users and the business itself, you should define a timeframe in which the test should pass or fail. That’s because you can’t keep a test alive forever.)
What is Changing?
I believe that there is not any “the best” idea in product management. All there is, is a better idea compared to previews ideas. This is because of a lot of reasons. You’d definitely have more data and information in the future about the product, market and users so you can make better decisions. What I mean by change is that every live and working product that cares about its users is always becoming better and better. Becoming better means implementing new ideas to create a better experience and value to the users. This is the job of a product manager as mentioned above. So you need to generate ideas test them and finally decide to make the change or not. Making a change could be bringing a completely new feature to the world or improving older features. I believe:
A product that is always changing is a product that goes under a lot of new tests and generation of new features and improvements.
The point is to have the changes after testing them. I mean not to bring a change without even being somehow sure about the effect and impact it can bring along. This little certainty comes with testing which is done in the first step.
Why should PMs test then change?
As said before, PMs are responsible for the failure and success of their product. This means you need to optimise everything to decrease waste. One of those things supposed to be your development efforts or in fact the human resource. We are doing tech so every ideas you have in your mind can be implemented but it needs to be developed and sometimes it takes a long time to develop some ideas. Therefore you need to be picky about what you want to develop and put into your product development cycle. Meaning that you need to test before going under the pressure of developing whole of a thing. This is why you need to test or sometimes in fact you need to run several tests before deciding to create the whole feature.
Testing helps you reduce your costs, reduce your risks, prevent users being shocked, etc.
To be honest, testing must be part of your product life-cycle.
Let’s look at an example:
Let’s look at one of my startups which is a place for amateur guitarists to learn and enjoy playing guitar. Laminor is a 3 years old startup which has more than 3000 UGC guitar chords which are free for users. We have implemented a freemium model in which users can consume some of the extra functionalities or content made available to them then pay for full usage.
This example is about changing the registration method on the website. Users could register via their Google account. We wanted to have more from them and were thinking about changing the registration method to mobile phone number verification. It was a huge change. We were into completely removing Google account. But it was a big change and we were a little scary. So what to do? It’s a major step. We decided to TEST then CHANGE!
The Test then Change process:
- A/B testing the phone number verification method and Google account method (There wasn’t any code behind the verification and we just faked it. We wanted to see if users are willing to enter their phone number in the registration step or not. Users saw a welcome message after entering their phone number. There wasn’t any verification SMS, just a simple digit validation on the website.)
- Comparing the results after 1 week of running the test (We saw that almost all users enter their phone numbers. We were happy for the first half of the test week because everybody was entering the phone number until we saw some users being stopped in the registration step. They were from other countries and we just showed our country sample phone number as the hint in the text field. So they gave up because they thought they wouldn’t receive the message. We added another hint to make sure they are not blocked. The results were awesome because everyone was entering her phone number.)
- Development just began after making sure we are not making a bad experience or something users dislike.
Did you see how we reduced the development effort and risk? Spending 1 week TESTing the idea then adjusting the test and finally deciding to CHANGE!
So, Test then Change!
Share in the comments below if you have similar stories within your products.