Docker Selenium Grid

With great test suites comes great responsibility. Responsibility of executing the Suits swiftly and reliably. In today’s fast paced world of CI-CD nobody wants to wait for hours for regression suites to execute — Enter “The Docker Selenium Grid”.

While the world reaps the benefits of Docker why should the testing community be left behind?

History: What is the Selenium Grid?

A group of VMs/Physical machines used to run selenium scripts in parallel to reduce execution time. One of the machines acts as a ‘hub’ while the others are ‘nodes’. The execution happens on the hub which distributes test cases across the nodes for execution. So for instance, a test suite with 1000 test cases executed on a single machine will execute all cases in sequence however, if executed on a Selenium Grid with four nodes 4 test cases will be executed in parallel which should ideally bring down the execution time to a fourth of the original time.

Grid Architecture

The Need for a Dockerized Solution:

As the size of the test suite grows a un-scalable system like a Selenium Grid remains of little use and becomes a nightmare to maintain. Test Suits spend a majority of the time waiting in queue for their turn to execute and because of such heavy loads the system makes the scripts flaky — rendering the script executions useless. One solution could be to have multiple Selenium grids in place — Bug Alas! there are drawbacks This approach requires a lot of compute and Secondly, maintaining such a vast infrastructure is a uphill task. Both these factors mean that the costs are sky high.

A Docker selenium grid on the other hand is a “On Demand Scalable” system. Which in lay man terms means –

- A grid will be created only when a Test Suite is ready to be executed. — Execution on fresh grids means scripts are not flaky.

- Several Grids can be created if multiple jobs are to be executed. — No Time wasted in waiting in a Queue.

- Each Grid created is unique terms of number of nodes. A larger Suite might run on 20 nodes while a smaller one uses just 4 Nodes. — Execution time reduced drastically.

- Once the script executes successfully, the respective Grid is also killed — No Open Processes, no temp files means maintenance is effortless.

- Using the configuration required to run one Selenium Grid (with 4 Nodes) we can have 4 Selenium Grids running in Parallel (with 8 nodes each) — Saving on total cost.

- One Click implementation. Triggering a Jenkins job does all the work — Building Docker containers, executing scripts, sharing reports, disposing containers.

With this solution the Test Suite with a 1000 cases (already viewed example) can be executed on a Grid scaled to 40 Nodes — Ideally reducing the execution time 40 times.

That Sounds Great! How Can I setup a Dockerized Selenium Grid?

Step-1: Setting Up Docker on Server Machine:

The steps to install Docker can be different based on the OS you are using. Steps to Install docker can be found at :- https://docs.docker.com/install/

Step-2: Downloading Images form Docker Hub:

Once Docker has been installed, let’s look at how to download images and run them to create containers.

Note- make sure your user is added to the ‘docker’ user group.

# Go Ahead and execute “$docker ps” → This should display the running containers on the machine.

Since we have not run any image yet , there should be no container running on our machine.

# Now to download “selenium grid hub” image from docker hub execute “$docker pull kandeel/ngtesthub” → This will download the image from docker hub to your machine.

# After Image has been downloaded , we can verify its presence in our local image repository by executing :- $docker images

# Similarly download a node image, execute :- $ docker pull kandeel/ngtestnode

# Once Hub and Node images have been downloaded , we can start creating containers from them. Just Execute the following Command :-

$docker run -itd -p 4444:4444 — name selenium-hub kandeel/ngtesthub

Step-4: Running Selenium Grid Hub Image

Now execute $docker ps to view the running containers and you will see that our Selenium Grid Hub has started running

We can see that our hub is running on 4444 port Number which has been mapped to Server Machine’s 4444 Port Number

Open Broswer and navigate to http://servermachineIP:4444/grid/console and you can see that selenium grid is up there with no nodes attached

Step-5: Running Selenium Grid Node Image

So , our selenium grid hub is up and running , now we just have to attach node to it , To do that , execute the following command :-

$ docker run -itd — link selenium-hub:hub -v /dev/shm:/dev/shm kandeel/ngtestnode and verify by executing $ docker ps command.

Open Browser and navigate to http://servermachineIP:4444/grid/console and you can see that selenium grid is up there with no nodes attached

Step-6: Building Selenium Grid Clusters Automatically Using Docker-Compose

A Docker Compose file allows us to create the selenium automatically without having to run several commands every time.

A Sample Compose FIle

Step-7: Configuring Jenkins for Docker Grid Execution

Integrating with Jenkins ensures that your team will get a plug and play solution, all the work of setting-up, executing and eventually disposing off the Grid will be done behind the curtains and all your team had to do is trigger a job — A single mouse click .

Things to keep in mind while configuring your Jenkins job:

1. Add Four String Parameters one by one namely: a)DockerPrefix b)NodeScale c)gitRepoUrl d)gitBranch
2. Go to Build tab and Add a step and choose Execute Shell, Add the below commands to the three “execute shell” steps one by one .

a) rm -rf /home/jenkins/{ProjectFolderName}/* && cd /home/jenkins/dockergrid && docker-compose -f $Binding.yml -p $Docker_Prefix up -d && docker-compose -f $Binding.yml -p $Docker_Prefix scale node=$NodeScale

b) docker exec $Docker_Prefix’_selenium-hub_1' /bin/bash -c ‘sudo chmod 777 -R /home/seluser/.m2/ && cd /home/seluser/AutomationProjects && git clone http://yogit.thakral:password@'$Git_Repo_Url' -b ‘$gitBranch’‘

c) docker exec $Docker_Prefix’_selenium-hub_1' /bin/bash -c ‘cd /home/seluser/AutomationProjects/{ProjectFolderName} && export nodecount=’$NodeScale’ && mvn test -Ddrivertype=’$drivertype’ -Denvironment=’$environment’ -Dprojectname=’$projectname’ -Dplanname=’$planname’ -Durlstr=’$urlstr’ -Djava.awt.headless=true -Dencoding=cp1252 -e‘

3. Go to post Build Actions and Add two post build tasks one by one and write the below commands in the post build task

a) cd /home/jenkins/ && mkdir -p $projectname && cd $projectname && docker cp $Docker_Prefix’_selenium-hub_1':/home/seluser/AutomationProjects/$projectname/target/surefire-reports/testng-results.xml . && docker cp $Docker_Prefix’_selenium-hub_1':/home/seluser/AutomationProjects/$projectname/finalReport/ .

b) cd /home/jenkins/dockergrid && docker-compose -f $Binding.yml -p $Docker_Prefix down

4. If you are using any HTML report api like Extent Report in your framework , Add a post build action as “Publish HTML Reports” and provide the source directory and file name in it.

5. For obtaining TestNG results into Jenkins Dashboard , Add a post build action as “Publish TestNG Results and Provide the “testng-result.xml” filepath in it.

Save these changes and now experience the power of The Docker Selenium Grid first hand.

Happy Execution :) !