The nightmare that is maintaining automated Selenium tests, is a nightmare. — a very drunk John
Forget the fact that I’m quoting myself, this statement holds very true. Even more so with the ever evolving landscape for web applications. But before we dig any further into the hell that is automated testing, let’s explain what Selenium is to those just coming across this malevolent tool.
Selenium is an automation ecosystem allowing developers to create end to end tests for their websites and web applications, as well as enabling other services such as web crawlers. Selenium was originally developed in-house by developers over at Google, and later polished and released by Thoughtworks as the most well known and used implementation “Selenium Webdriver”. The idea behind Webdriver is quite simple; an official api is maintained by the Selenium team, you download bindings for your favorite programming language (C#, Python, Java, etc.), you acquire a webdriver for the browser you would like to test against, and finally you have your selenium bindings talk to your webdriver, which talks to your browser?
Can you paraphrase what you paraphrased?
- Selenium maintains an official REST api which webdriver maintainers must adhere to so our Selenium bindings work universally across any browser.
- Selenium also maintains the official selenium bindings (or library, module, what have you) for Java, C#, Ruby, Python, and Javascript. There are also unofficial bindings being developed by the community such as webdriver.io and Nightwatch.js which simplify rudimentary tasks, and give you a more opinionated structure for bootstrapping your test cases.
- Your selenium bindings aren’t pulling any magic, they’re just communicating with your webdriver via a REST api.
So, whats Selenium’s role in an application’s lifecycle? In companies with a dedicated quality assurance department, it’s common place to have developers maintaining automated tests for a continuous integration and continuous delivery. The idea being for example, once a pull request goes through and a build is triggered, these automated tests will run against the application before it’s deployed — or even before it’s merged.
Okay, cool. Sounds straightforward, why were you freaki-
Oh how naive you are, reader.
The largest challenge with Selenium doesn’t actually lie in writing the tests themselves, but infrastructure. Needless to say, maintaining the tests is a terrible pain, but laying out a dynamic infrastructure capable of scaling at the pace of development is it’s own hell.
At my current company, we tried to tackle this problem by incorporating containers into our stack.
We unit test our web application (both our frontend and backend services), but we also roll out our Selenium tests to ensure our business logic is correct and our web application is functioning as the consumer would expect. For that precise reason, we leave our end to end tests to our QA department rather than development.
Our Selenium codebase is dockerized and our containers are provisioned to run only tests pertaining to the changes made to front end. This workflow is perfect for our CD environment, using Concourse CI to manage our triggers and pipelines. By shipping our tests in containers, we don’t need to worry about the actual deployment environment. If we ever need to nuke a server node, we can just add a new docker daemon to our swarm and onwards. We’ve also been experimenting with lambda servers — which drastically reduces our workload. Because we’re using a selenium hub, it doesn’t actually matter where our test is being executed from.
So what’s a Selenium hub?
The Selenium hub/node architecture allows us to farm our tests to remote browsers on different environments. This is especially helpful for a number of reasons, the most prevalent being that we can unilaterally test across different environments and browsers without any severe overhead. The selenium hub allows us to map our selenium nodes (independent machines executing our tests), physical or virtual, to a single front facing api that our bindings then use to instantiate remote webdriver instances.
Enough about infrastructure! Where do I get started writing my own te-
Patience! Writing tests with Selenium isn’t as simple as it may seem. Or I should say writing stable, maintainable tests. You can take the following with a grain of salt, but you aren’t going to produce results if you don’t have basic web development experience. Even then, tossing an established Java, Python, or .NET developer at the task isn’t going to cut it either. What webdrivers do under the hood with the actions fed to them by your bindings or hub/node, is simply execute Javascript. There’s no voodoo going on here, your webdriver isn’t invoking any win32, osx, or linux api calls. It’s just executing Javascript. It also doesn’t hurt to understand how your bindings are interacting with your given application and what’s going on under the hood.
Then what’s the point in using any bindings aside from Javascript?
Take a breath grasshopper, I never said you would be exclusively writing Javascript. Knowing Javascript and it’s role with websites and web applications is imperative, yes. But the the Selenium bindings will cover a good margin of your use cases. It’s only when you start poking around with more modern web applications developed with Angular.js, Angular, React, Vue.js, etc. that you start encountering counteractive behavior between what your bindings should be doing, and what they’re actually doing. In these scenarios, the typical cause is usually an unwanted event being dispatched to an element, or an event that was supposed to be dispatched bubbling into it’s parent elements.
So what do we do in these edge cases?
Code! Code! Code! No but really, as you start attempting to pull off more and more complex user interactions with Selenium, you’ll find yourself at a loss of support for the official Selenium bindings. I’ve experienced this myself, and ended up writing my own Javascript bindings pyselenium-js.
No but really, where do I get started writing tests?
I’m actually going to be covering this in a follow up post. What I wanted to emphasize here was the practicality and the implications to setting up end to end test for larger, more modern web applications — as well as cover the biggest drawbacks to large scale selenium testing and my solutions from personal experience. In my next post I’ll cover best design practices, setting up testing environments, and how to account for scaling.
- slightly sober John
