Product Management for Agile Businesses: Validation

The validation phase of the Product Management Lifecycle is one of its most critical steps. Without solid validation, a business can spend scarce resources developing the wrong offering and missing opportunities in the market. Suffice it to say that many business failures occur due to poor validation.

Validation of a signal (an idea for a new feature or offering) will begin once that signal and been deemed a candidate for validation. A well structured signal will contain some concept of:

• what is being requested,
• the problem or opportunity the signal is expected to solve, and
• the value that addressing that problem or opportunity will generate.

This concept will be further investigated during the validation phase.

What is the goal of Validating a Candidate?

There are multiple objectives in validating a candidate. The outcomes from this process are:

• Ensure that the problem has been well defined and that the value to be generated is well defined. Value should be captured quantitively where possible, for example, by saving 4 hours per transaction on average.
• Understanding who would benefit from the candidate? What proportion of current customers and prospects.
• Trying to measure the revenue and profit impact for your business, especially if it appears that a large investment will be necessary.
• Understanding what strategic benefits can accrue from the candidate. For example, does it increase barriers to exit for adopting customers.

Note that during the validation phase we not not trying to validate the solution for the candidate, indeed at this point we do not and should not even have a solution.

The Size of a Candidate

Understanding the expected size of a candidate is important at this stage uniquely to evaluate how much validation should be done. However, it is not recommended to go much beyond a very small, small, medium, large or very large scale — given the nature of the solution is unknown, moving beyond this would be non-sensical.

For example, if the candidate is addressing small changes to the offering, for example adding fields into a form, it may not be sensible to invest much in the validation effort. If, on the other hand, the investment is sizeable, a new module is required, then more validation should be done.

The product manager should apply good judgement and be prepared to defend why they have chosen what level of validation, and what tools is use during the the validation.

Validation Methods

There are many methods that can be used to validate candidates. The product manager should select those that are appropriate. For certain methods, it may make sense to bundle validation efforts across multiple candidates.

The appropriate validation methods will also depend heavily upon the type and size of your business. If you’re B2C versus B2B, whether you have a very large or small client-base, what kind of clients and verticals you are servicing.

• Client calls: reach out to a representative selection of your clients to understand how a candidate may impact their business. It may make sense too to bundle multiple candidates together when discussing to make the most use of their time. It’s essential to go in well planned and make sure you get an idea of the value a candidate can generate. Can you also get a quantitive idea? Note that for interviews, it’s recommended to reach out to a minimum of five customers, however eight to ten is great and twenty is fantastic.
• Target-business calls: for some candidates, the goal will be to expand the business out to new market segments or new geographies. In this case, you would want to reach out to these target-businesses in those markets to investigate the value of the candidate. Note that it is not recommended to reach out to business currently in the sales cycle to avoid disrupting that cycle and the messaging from the sales organization.
• Surveys: surveys are great when you have a larger number of clients, or have several relatively low value candidates that you want to get a rough idea of value. When you ask about interest in a candidate, don’t just ask clients whether they are interested in the idea, ask them on a scale of 0 to 10 how much difference a candidate would make to their business. You should also ask for free-form comments.
• Customer Advisory Boards: having a customer advisory board is a great way of assembling your key clients and enabling them to discuss together your candidates. Oftentimes the discussions between clients will enable you to go even further down value discovery than with just one client.
• Focus Groups: similar to customer advisory boards, can you get together a set of target users to discuss the candidate or a set of candidates to get their feedback?
• Data: what data can you get from your systems that could indicate whether a certain new feature might be highly valuable? This can be flows through the application, time spent doing activities that you could remove (and thus put a dollar value on) etc. Data from systems is probably the principal way of determining value for B2C offerings.
• External Data: what industry trend data can you use to show the value of a candidate or on the other hand show that a candidate has little value?
• Short business case: at this point, if the candidate is looking like it will require a significant investment, it might make sense to do a back of the envelope style business case to understand what is the potential financial outcome.

Validation Outcomes

Once the validation process has been completed, write it up in a short report — try to keep it to one page so that it can be easy consumed by stakeholders. Use of the validation outcomes to work through the scoring and prioritization process to understand how important this particular candidate it and where it will fit in amongst the initiatives that are currently being worked upon.

Indeed, the validation exercise may lead to the conclusion that any further work on this candidate is not warranted.

In case it does, it’s essential to tie back the validation to the objectives introduced in earlier in this article.

Value Metrics

Coming out of validation, it should be possible to determine initial success metrics for the candidate, in order to answer in a quantitive way “what does success for this candidate look like.” Depending on the feature, it could be a number of uses per month, it could be financial gain, time saved etc.

Ideally, it would be possible to tie the metric back to direct success at the client. The client saved or generated a certain amount of money — however this is often difficult to calculate directly without significant investment and participation by a client and they are unlikely to find this worth their while. Hence, look for a proxy figure to represent this in your metrics.

You may be asking whether it’s not too early to be defining these metrics. The answer is a resounding no — thinking through metrics will confirm whether you can understood the value that the candidate is expected to generate, and you’ll need your metrics defined as you go into the solution phase of the product management lifecycle.

The Story

Whilst not mentioned as an explicit outcome, by the time the validation phase is complete, you should in a position to tell a story about why the candidate is valuable.

Hence, be in a position to say if we do this, it will have this outcome for the customer and therefore that outcome for our business.

Conclusion

Running a validation process is the key part of the product management lifecycle. It must focus on understanding value and preferably being able to quantify it.

Without great validation, it’s hard to run a successful Scoring and Prioritization process.