Responsibility #8: Encourage Continuous Testing

By Michael Colemere and Candianne Haacke

Benefits of Continuous Testing
Product managers use continuous testing to deliver quality products at an accelerated rate while improving user experience. The notion of continuous testing is rooted in agile development methodologies. It suggests that testing should occur throughout the product lifecycle instead of waiting until very late stages. This will reduce development churn and increase user acceptance.

Three Simple Steps to Getting Started
Continuous testing is a principle that can quickly become overly complex. To keep things simple, product managers might consider incorporating the following three practices.

First, foster a culture of ongoing data-driven decision-making (instead of making landmark decisions based exclusively on personal preference or intuition). One way product managers can achieve such a culture is by requiring the product development team to continuously seek user feedback.

Wise product managers will take the time to actually know what users want instead of assuming they know what users want. User testing methods include:

· Collecting and reviewing user-generated feedback (using form submission fields)
· Intercepting users currently on-site with true intent studies (using Qualtrics or Loop11)
· Identifying new opportunities with focus groups (using a facilitator or eye tracking software like tobii)
· Organizing key themes with affinity diagrams
· Evaluating findability of content with tree testing
· Determining market performance with the net promoter score
· Measuring promotional strategies through multivariate or A/B testing (using Optimizely)

Second, encourage quality assurance to lead out during iteration planning meetings. When quality assurance leads iteration planning meetings they demonstrate to product managers whether or not a user story is complete. The quality assurance portion of iteration planning meetings keeps developers accountable for their work. This means that the quality assurance engineers also require a bit of air cover from business leaders and earned trust from their peers.

Third, support organizational efforts to adopt enterprise tools. Third-party tools like Qualtrics, Loop11, and Optimizely can be instrumental to a product manager’s success. Because these types of services are most cost-effective at an enterprise level (rather than as one-off requests), product managers can collaborate together to encourage their use in ways that help the entire organization succeed.

Tom Kelley, the general manager of IDEO, encourages employees to “fail often so you can succeed sooner.” Failure can be a scary feeling, but if it occurs at the right time, disasters can be avoided. Continuous testing is a method of identifying failures (or opportunities) early on so that quality products can be delivered faster and better. Continuously seeking user feedback, empowering quality assurance, and supporting enterprise-level solutions can accelerate product development while ensuring a quality user experience.