The nightmare that is maintaining automated Selenium tests, is a nightmare. — a very drunk John
Forget the fact that I’m quoting myself, this statement holds very true. Even more so with the ever evolving landscape for web applications. But before we dig any further into the hell that is automated testing, let’s explain what Selenium is to those just coming across this malevolent tool.
Selenium is an automation ecosystem allowing developers to create end to end tests for their websites and web applications, as well as enabling other services such as web crawlers. Selenium was originally developed in-house by developers over at Google, and later polished and released by Thoughtworks as the most well known and used implementation “Selenium Webdriver”. The idea behind Webdriver is quite simple; an official api is maintained by the Selenium team, you download bindings for your favorite programming language (C#, Python, Java, etc.), you acquire a webdriver for the browser you would like to test against, and finally you have your selenium bindings talk to your webdriver, which talks to your browser?
Can you paraphrase what you paraphrased?
- Selenium maintains an official REST api which webdriver maintainers must adhere to so our Selenium bindings work universally across any browser.
- Your selenium bindings aren’t pulling any magic, they’re just communicating with your webdriver via a REST api.
So, whats Selenium’s role in an application’s lifecycle? In companies with a dedicated quality assurance department, it’s common place to have developers maintaining automated tests for a continuous integration and continuous delivery. The idea being for example, once a pull request goes through and a build is triggered, these automated tests will run against the application before it’s deployed — or even before it’s merged.
Okay, cool. Sounds straightforward, why were you freaki-
Oh how naive you are, reader.
The largest challenge with Selenium doesn’t actually lie in writing the tests themselves, but infrastructure. Needless to say, maintaining the tests is a terrible pain, but laying out a dynamic infrastructure capable of scaling at the pace of development is it’s own hell.
At my current company, we tried to tackle this problem by incorporating containers into our stack.
We unit test our web application (both our frontend and backend services), but we also roll out our Selenium tests to ensure our business logic is correct and our web application is functioning as the consumer would expect. For that precise reason, we leave our end to end tests to our QA department rather than development.
Our Selenium codebase is dockerized and our containers are provisioned to run only tests pertaining to the changes made to front end. This workflow is perfect for our CD environment, using Concourse CI to manage our triggers and pipelines. By shipping our tests in containers, we don’t need to worry about the actual deployment environment. If we ever need to nuke a server node, we can just add a new docker daemon to our swarm and onwards. We’ve also been experimenting with lambda servers — which drastically reduces our workload. Because we’re using a selenium hub, it doesn’t actually matter where our test is being executed from.
So what’s a Selenium hub?
The Selenium hub/node architecture allows us to farm our tests to remote browsers on different environments. This is especially helpful for a number of reasons, the most prevalent being that we can unilaterally test across different environments and browsers without any severe overhead. The selenium hub allows us to map our selenium nodes (independent machines executing our tests), physical or virtual, to a single front facing api that our bindings then use to instantiate remote webdriver instances.
Enough about infrastructure! Where do I get started writing my own te-
So what do we do in these edge cases?
No but really, where do I get started writing tests?
I’m actually going to be covering this in a follow up post. What I wanted to emphasize here was the practicality and the implications to setting up end to end test for larger, more modern web applications — as well as cover the biggest drawbacks to large scale selenium testing and my solutions from personal experience. In my next post I’ll cover best design practices, setting up testing environments, and how to account for scaling.
- slightly sober John