Product Validation: Achieving the Goal
Isn’t it useless spending your time creating a product that nobody wants or needs? How can you ensure that the product you’re making will be useful and meaningful?
One of the most important things, when you are making a product, is that validating that there is a demand for your product itself (Kular, Nimi, 2018). The product ideally should attract real users and also the stakeholder that has an interest in the product. It will be very devastating when you have launched your product, only to realize that there are very little to none interest from people who are your target customer.
In designing software, for example, the software that has been made by you and your team will be utterly useless if the real users and the stakeholder don’t want to accept your software. One of the biggest and most common mistakes that software engineering teams make is to have far more confidence in their product specifications than they should, and they move forward and think they’ll adjust the product — if necessary — once they get beta feedback (Marty Kagan, 2015). To ensure that this won’t happen to your product, you have to do, what people call, Product Validation.
According to Alia Bonetti (2015) in NASA Systems Engineering Handbook, Product Validation is a Validation is performed for the benefit of the customers and users to ensure that the system functions in an expected manner when placed in the intended environment. The validation proves whether “the product was built right”. It also generates evidence as necessary to confirm that the product meets the capability and operational expectations of the customer and any other stakeholder.
There are many ways to avoid major validations upsets. Your requirements must be well-defined. Ensure that each end product in the system structure was correctly realized in accordance with its specified requirements before conducting validation. If validation is left to be done too near to the end of a project’s life, major discoveries can result in serious cost, performance, and schedule issues.
This is a list by Alia Bonetti (2015), showing the inputs and outputs of a product validation process:
The Inputs
- Verified Product, you can use a Verification Report Matrix as I provided below:
- Validation mission plan
- Customer expectations, such as Concept of Operations, Mission Objective, Profile, etc
The Outputs
- Validated Product
- Discrepancy reports/notes and necessary corrective actions
- Validation reports
Now that I have covered about the inputs and outputs, how about the validation process itself? Here is the Product Validation Process chart from NASA Systems Engineering Handbook (2015):
Basically, there are 4 steps in the product validation process:
- Validation Planning → Determine the type of validation that must be used for the particular aspect of the project that is being validated. You also should review the mission plan with the customer in order to confirm that the proper end product is being validated in the required environment and under the specified operating instructions
- Validation Preparation → You should gather: validation plan (procedures for each type of validation, objectives of steps within the validation process, criteria for success vs. failure, and well-defined requirements to compare against validation), product to be validated, set of customer expectations, and any of the supporting resources.
- Conduct Validation → Follow the procedures from the validation plan, take any necessary measurements, and record resulting data needed to determine whether or not validation was successful. After conducting the product validation, you should be able to determine: a. Supporting information to confirm that appropriate results were obtained; b. Whether the products comply with customer expectations; c. Whether the product was properly integrated with the validation environment according to customer expectations; d. Whether the product functions with any required interfaces
- Analyze Results → You should: a. Analyze data for quality, correctness, integrity, and consistency; b. Compare the actual validation results to the expected results; c. Identify any deficiencies that arose in the validation process; d. Determine corrective actions needed to address any deficiencies; e. Record all analysis data and prepare to repeat validation as needed
Important Notes:
- Each product in the system (think about the Product Breakdown Structure → hierarchical structure of things that the project will make or outcomes that it will deliver) needs to be validated to confirm that it meets customer expectations before it is integrated into a higher level product.
- If a deficiency is discovered during validation, be careful that its correction does not create a new issue with a product that previously operated without issue. “Regression testing” is an ideal method that should be used to deal with this issue.
- Documentation is essential throughout the validation process in order to have proof that customer expectations have been met.
What have we done in our project about Product Validation?
Well, we are lucky that we get to meet our user (the stakeholder) once every two weeks. In every meeting, we consult about our software and ensure that the software that we are developing is in line with the user’s need. The user (luckily) also has been very supportive towards us.
In every sprint review (read my other blog: Agile Software Development), our user always try out the progress that we have done on the last sprint. Then, our user gives us feedbacks about what he thinks about the product that we’ve been making. He can either accept it, reject it, or accept it with a proviso.
We also suggested some of our inputs to the product owner. For example, the User Experience type. once we have to input our password when we wanted to change our profile picture. We thought that it’s not necessary, so we asked him to consider removing the password field. And he did like our suggestion and then removed the password field from the wireframe.
Conclusion
It is important to give users the product that they want or need. Sometimes, what people’s desire cannot be guessed just from the perspective of the product team. Therefore, a Product Validation is needed to ensure that the product that is being made is useful and meaningful for the real users.
Reference:
- Morisette, J. T., Baret, F., Privette, J. L., Myneni, R. B., Nickeson, J. E., Garrigues, S., … & Kalacska, M. (2006). Validation of global moderate-resolution LAI products: A framework proposed within the CEOS land product validation subgroup. IEEE Transactions on Geoscience and Remote Sensing, 44(7), 1804–1817.
- Alia Bonetti (2015) NASA Systems Engineering Handbook for CSULB EE 400D
- Marty Kagan (2015) Product Validation from https://svpg.com/product-validation/
- Nimi Kular (2018) How to Validate Your Product Ideas
