Is Selenium still the right choice for Test Automation?

I am a test automation engineer who had been working with Selenium automation library for the past 10 years. I had been watching the rapid adoption of Cypress automation tool for the past couple of years. Initially, I thought, it is just one of the other competitors of Selenium and tried to ignore all the good things people say about it. I always tried to find it’s negative sides. It can run tests in Chrome browser only. It cannot work with multiple browser tabs. Iframe switching is nearly impossible without a plugin etc…

I did not want to move away from Selenium. Naturally being lazy, I did not want to learn Javascript or try a new automation tool. I was happy with Selenium and Java.

Whenever I talk with my developer or tester friends, they would talk excitedly about Cypress. It is so easy to set up. The test runs are so stable. You do not have to worry about flaky tests anymore. I looked at my automation suite. I have around 500 UI tests running daily. The pass percentage hovers around 80–85% and I am trying my best every day to fix the flaky tests. When I fix some, others would start failing. Test maintenance became an everyday chore. When I talked to my friends, they said, they don’t have to worry about flakiness because Cypress would manage the wait by itself. It was so hard on Selenium. Some elements would take longer to load. There is an incessant spinner that pops out every now and then and causes ElementClickIntercepted exception. My automation was a mess. I started dreaming about moving to Cypress.

I started studying about Cypress. Soon the reality struck me. My application has got countless iframes. It is a vendor-provided ERP system. Wherever I click, it would open a menu tab with a new iframe. I could count up to six of them during my manual testing. The spinner would not go if I use Cypress. I would still need to handle it. Worse of all I have to work across multiple browser tabs which Cypress is not able to do. I have no option but to stick with Selenium for a while.

Then I decided to look at Selenium documentation again to find a solution to my problems. Though it was glaring at my face all these years, this time it struck me like a lightning.

Selenium automates browsers. That’s it!

What you do with that power is entirely up to you.

Yes! Selenium just automates browsers and the rest is up to us. No false promises. No promises about automatic waits. No ajax session handling. It would do nothing to solve your flaky test problems. It is all up to us. We need to handle it. But I did not want to solve all these problems myself. I did not want to re-invent the wheel if there is one already existing. I found a few good frameworks. Tried a few of them and found that they would solve the wait problem for me. But the wait problem I had already solved myself up to an extent. I could use explicit waits to solve my wait issues. Again I have to deal with spinners, elements with varying load times, and then those nasty dynamic iframes as well for which none of these frameworks offered a solution.

Finally, I thought the solution may lie with my understanding of the problems. So I decided to look at them and started to think about a solution.

First problem is the iframes. I had to write countless lines of code-switching back and forth between iframes. A lot of code bloating could be avoided if I could remove those switch commands. Then I found the solution. Why can’t I cycle through these iframes to find the element? I created a function to do that. Then I wrapped it with a wait function. So the iframe cycling will continue until it finds the element or until the timeout happens. The problem is solved. It worked seamlessly. I no longer have to write driver switch commands over and over again.

Now, I have to solve the spinner issue. This nuisance spinner would occur during page loading or menu switching or whenever it feels like. It had a mind of its own. I had to write a script to wait until this spinner disappears. Still, I would get ClickInterceptedException as the spinner would pop up every now and then. Then I thought, why should I bother about this exception? I can easily catch it and retry again. I did exactly that. The spinner issue disappeared. I no longer had to add a script to wait for the spinner to disappear. I refined it and added a similar technique for other user interactions like text input as well.

I used to be a fan of dependency injection frameworks but came to hate the telescoping constructors I had to create sometimes. I was using cucumber pico container. I did not want to use it anymore.

I created a small framework with my custom functions and decided I would try to run them through static functions. Static functions and parallel runs will not go well with each other. Then Java Threalocal class saved me. I could implement parallel runs while my framework APIs are static. I gave it to my colleague to experiment with a small project.

Fast forward two years and many iterations of that framework with the addition of many user-friendly APIs, we are currently running around 4000 large integration tests per day with negligible flaky tests and many projects passing at a 100 percent pass rate. We have three different ERP systems that talk to each other and the UI tests cover all these applications. Oh, I forgot to tell, my company migrated all their automation project to this new framework.

We eliminated the need for dependency injection. No more constructors. All method calls are static because the framework will handle the concurrency. Testers do not have to handle waits, spinners, or iframes. Just automate test scenarios and the framework will handle it. We are a happy team of test automation engineers now. We are churning out hundreds of stable automation tests per month.

The magical code is similar to this,

find(“locator”).click();

That is all it wanted. Nothing less, nothing more. Even the locator is passed as a string which the framework would parse to corresponding By locator of Selenium.

I am happy that I stayed with Selenium. It is an open-source tool. It is not freemium software. It is a powerful tool and it is up to us what do we do with it. I am not saying that everyone should create their own framework to deal with stability issues with Selenium. You can use Selenide. It is an awesome framework. There are other wait libraries like ‘Awaitality’. Use any one of them. Your tests would be stable. It is up to us what we do with Selenium.

I would like to hear about your experience with other automation tools. How is your experience with writing large integration tests with Cypress or other tools? Do you have thousands of tests running daily with a CI/CD tool with any of these tools other than Selenium?