Run Selenium UI tests in Docker containers

5 min readDec 30, 2019

Along with Unit and Integration Tests, UI tests are very important in the software development lifecycle. They can detect bugs and anomalies before going to Production. But, UI tests could be so expensive to setup: we need a full VM with all the dependencies. And sometimes we need more than one VM if we want to scale out to run tests in less time.

What would be a light and easy solution to run UI tests ?

The short answer

Instead of relying on VMs, we can launch UI Tests in Docker containers !

The answer in a video

Let’s see how that works through a demo.

We will take a sample .NET Core UI tests written using Selenium. Then, we will setup and deploy the Selenium containers to run the tests.

The principles still apply to other programming languages and frameworks.

What is Selenium ?

Selenium is an opensource framework for developing UI tests can run on Chrome, Firefox, IE and Edge.

When we deploy an app, we want to test it manually. A tester will open the browser, paste the URL, type his login and password, click login button and check if the home page shows up correctly. Selenium comes with the promise to automate this process. A robot will do that instead of a human ! The following video explains it in details.

Web UI tests using Selenium

1. Setting up the UI tests and switching to RemoteWebDriver

We typically use the WebDriver class in the code to configure parameters like the required browser, incognito mode, download directory, browser driver, etc… By default, WebDriver assumes that we want to run the UI tests inside the host machine. But, here, we want to run them inside a docker container. So, we need to connect to the container IP address. For that, we need to use the RemoteWebDriver and specify the IP address and port number for the container (something like: http:localhost:4444/wd/hub).

var driver = new RemoteWebDriver(new Uri(<RemoteUrl>), options);

A sample UI test that opens a web page and checks that it contains a certain label would look like the following:

public void PageLive_Test()
{
var options = new ChromeOptions();
var driver = new RemoteWebDriver(new Uri(<RemoteUrl>), options);
driver.Manage().Window.Maximize();
driver.Navigate().GoToUrl("https://www.youtube.com/houssemdellai");
Assert.IsTrue(_driver.PageSource.Contains("Houssem Dellai"));
}

2. Setting up Selenium docker containers

We can run Selenium on single/standalone container or on a Grid of multiple containers. Let’s focus in more details in each approach.

2.1 Running Selenium in a standalone container

This option means that we can use only one single container to launch all the tests. This container is selenium/standalone-chrome or selenium/standalone-firefox. We are limited to only the resources available for one container and we should handle targeting Chrome or Firefox browsers. This option is the easiest, still good for small projects but it doesn’t scale.

To run the above test, we just need to run the container exposed on port 4444 which is the default port that connects to the web driver:

$ docker run -d -p 4444:4444 -v /dev/shm:/dev/shm   
             selenium/standalone-chrome:3.141.59-yttrium

Then we configure the RemoteWebDriver’s URL:

var remoteUrl = "http://localhost:4444/wd/hub";
var driver = new RemoteWebDriver(new Uri(remoteUrl), options);

And we run the tests using the .NET Core CLI from within the project folder:

$ dotnet test

At this point, we should see the results about the tests failure or success in the console.

To be able to visually view what is happening inside the container, we could use an image that have VNC preinstalled: selenium/standalone-chrome-debug.

2.2 Running Selenium in a Grid of containers

If the first option doesn’t scale, this one solves this problem. We can create a Grid of multiple and different containers, something like 10 instances of Chrome v79, 20 instances of Chrome v60, 30 instances of Firefox latest version. As a result, we can run tests in parallel against different versions of different browsers. They will be controlled by selenium/hub.

Selenium could be used to run not only web UI tests, but, also mobile UI tests.

The Hub becomes the central part where we should connect to. It will then dispatch the requests on its nodes depending on the requirements for each test.

2.2.1 Setting up Selenium Grid using Docker Compose

Because the Grid is a composition of multiple docker images, it is easy to configure it using Docker Compose:

Then we need to run the docker compose command:

$ docker-compose up

This will spin up the Grid. Now, we can connect to it through the remoteUrl and run the tests as in the previous section (2.1).

We are using .runsettings file in order to change the remoteUrl through environment variable without recompiling the app.

Test runned inside Selenium Grid containers

We can view the Grid dashboard showing the different connected nodes, as following:

Selenium Grid console showing connected browser nodes

2.2.2 Setting up Selenium Grid in Kubernetes

Docker compose is useful for development in local machine. But, when it comes to releasing the system into production, then we will typically need Kubernetes. What was achieved with docker compose is also achievable with k8s manifest files. So, we create the Deployments for the Hub, Chrome and Firefox, along with the Service that will point to the Hub.

We’ve exposed the Service type as LoadBalancer to get a public IP address.

We need to connect to a Kubernetes cluster and run the command:

$ kubectl apply -f selenium-k8s-deploy-svc.yaml

To get the IP address for the service, we need to run : kubectl get svc

Then, again, we should edit the remoteUrl to take the external IP address of the Hub. And finally we can run the tests with the same dotnet test command.

3. How to make it even better

There is always different ways to make things even better. For example, here we are still launching the UI tests from the local machine, even if they will run on a remote web driver on a container. So, it would be interesting to run it all in containers. We can also configure the Horizontal Pod Autoscaling in Kubernetes to scale automatically the number of replicas.