Product Testing — Challenges and Opportunities

Moving from an IT services company to a product team in my current company presented me with its fair share of challenges. When I joined the product team of Cantiz Nucleus, I was thinking and planning everything from the perspective of a typical services project. Everything I expected as a part of my testing activities was not in place or were in another form. Since I was working in service projects for a long 5 years, these things were quite frustrating for me. It took me nearly one month to clearly understand the fact that product testing is different from testing a services project. Experience is a hard teacher because she gives the test first, the lesson afterward. In this article, I will discuss the lessons I learned while testing Cantiz Nucleus IoT platform.

Ideation Phase
When you work in a product team, you actually own that product. An idea or a concept can dramatically change the product. The first meeting held during the ideation phase was expected to be just another meeting. But I came out of the meeting highly impressed and understood the fact that “Team has more knowledge, experience, and insight than an individual”. Entire product squad which includes product owner, developer, analysts, UX designers and configuration team members were present at the meeting. The wonderful thing I learned about the ideation phase was that it aligns the entire team around a common objective. Ideation phase became effective for the product once we made it a core part of the process of product delivery.

Agile was always a scary thing for me. The reason is quite simple. I was never a part of an agile team/process before and I didn’t like the word scrum due to some reason that can’t be published. For a tester like me, agile methodology helps in testing the product continuously across the sprint which immensely helped me in finding defects at an early stage rather than testing after entire development is done as in a classic waterfall model. We get many comments from product owner based on small incremental releases happening as part of sprint deliverables and those questions are helpful in the long run so that the credibility of the product features are not questioned by the marketing team or by the customers.

Automation makes testing easier. But a lot of effort was required for automation, especially for someone from a services background. Our product is built on a big data stack with a lot of headless components. Off the shelf automation testing frameworks and tools, most often was not an absolute fit for our requirements. Custom scripts for automation and load testing were prepared in scripting languages like Python and Golang. In a services project, we might be testing code provided by the customer or developers from our own team. In both scenarios, testability of the code and the component getting developed is never considered as a responsibility of the development team. For any successful product, automation testing plays a pivotal role in reducing regression testing effort and ensuring product quality, when new product releases are made. It’s important that the automation testers and developers work in tandem, ensuring the testability of the product and in the development of a test suite which has a very wide coverage of the product features.

Black & White
Black box testing and white box testing are some terminologies used in ISTQB. White box testing was always considered to be a developer’s responsibility. But after joining the Cantiz Nucleus, I clearly understood that here there are no absolute role definitions and organisational level boundaries for testers. Testers were also involved in white box testing, which helped me in identifying critical bugs at an earlier stage of testing itself. Even though white box testing is a challenge for a conventional tester, the learning from white box testing was simply splendid. White box testing improved my knowledge on product and helped me understand the product from its roots.

Never ending process
Well .. In a services project, we have an idea when the testing activities are going to end. But while working in Cantiz Nucleus, what I understand is that a continuous change is required in every product, based on the market research and analysis. Vexation about ever changing requirements was a concern for me at the beginning of the product lifecycle. But the clarity of vision and the long-term goals set by the product owner and the group meetings to explain these goals helped us set the right expectations in testing a product.

Tail end

“Product Testing is more or less doing continuous experiments on rats to figure out a solution whereas project testing is like providing the existing solution.” To do experiments, to make mistakes, to learn and relearn, being a part of the overall product evolution cycle are some of the expectations from a tester working in a product.