To Robot or not To Robot, that was the question

Months ago I switched projects and landed in a different ecosystem that challenged my comfort zone.

If you have been following previous stories, I used to work with Cucumber + Java to build a big and complicated framework that would help us run more than 300 regression tests in 3 different sites, in 2 different viewport sizes, which was part of the Continuous Delivery process. Later I came to a project where we barely had 5 test cases that were part of the Continuous Integration pipe. In Robot Framework.

This is an Instagram user featured in the client’s webpage using a bag sold by the client

My initial personal challenge was to bring Cucumber + Java framework to the new project. Then decided not to do it, for different reasons and learning lessons I want to share here with you.

1. Resources are important

My first instinct was to get rid of the Robot Framework that was being used and bring our own Cucumber + Java framework to the new client and demonstrate what it could do if we were able to implement it. Soon I realized that I did not had access to the same amount of resources.

See, with the past client, I had access to more than 1 Windows VM that could host 10 Chrome nodes, which meant that we could run 10 simultaneous tests. With this new client I had access to just 1 node, which could host only 5 nodes.

With the past client, there was a team of 4 automation engineers. Together, we were able to add new test cases to the regression, analyze the automation reports, and maintain the old test cases. With the new client, there were only 1 automation engineer and 1 functional test engineer. That meant that we would not be able to fully do all 3 activities without risking quality.

2. There is no one-fits-all solution

There were a lot of differences between clients. One had a lot of money to spent in the e-commerce solution, the other was skeptical on wether they would be able to do money with e-commerce. One had a full in-house IT department working in the solution. One had a small team of subcontractors working on it.

But most important, from automation point of view, was the Continuous Delivery process in place and working. The new client is still working with major monthly releases. Which means that each release would have it is own test strategy and plan, since the majority of changes will be concentrated in an specific area of the system.

At the end I realized that the big and complicated yet powerful framework would not help this new client. While we want help both clients to be successful, one client next step was to implement CD practices, while the other client next step was to redesign the current solution. Different problems, different solutions.

Contrary on what all-purpose gasoline can do, I learned that there is no one-fits-all solution

3. The tool is not the limit

Then the goal changed. From “bring a new framework”, to “grow their framework”. As a team, we agreed on starting adding automated test cases to ease the manual functional regression testing done in other areas of the system.

During this phase, there were multiple times when I hated Robot Framework because I was not allowed to easily do what I wanted. It took me a while to realize that I was trying to follow the same paradigms in Java than Python. I just needed to tweak how I was coding and the next test cases flowed.

At the end, we were able to deliver 2 important releases to the client, both with significant amount of redesign, we were able to increase the number of test cases in the regression suite and share the results of the execution. The client had a record number of orders this past holidays.

New problems, new solutions.