Web Automation — How to set up Selenium so that it easily runs on your Computer, Docker containers, and Bitbucket Pipelines.

Those who have been into web automation are aware of Selenium. Selenium is a potent and valuable tool for webpage automation. It is mainly used for webpages testing as well as automating simple web tasks.

Under Selenium, there are two main components:

1. WebDrivers.

WebDrivers are what Selenium uses to communicate with the browsers. Each browser has its specific WebDriver. For example, Firefox uses GeckoDriver, Chrome uses ChromeDriver, Opera uses OperaDriver, etc. To learn more about WebDrivers you can check out official Selenium documentation about WebDrivers here.

2. Language Bindings.

These are some official languages that Selenium supports. For example, Java, Python, JavaScript, C#, etc. So, language bindings are the code that Selenium provides to developers so they could use to interact with Selenium. For this demonstration our language binding is Python.

Recently I was helping out a client to automate website test cases with Selenium and Python. Through the process, I came across a challenge where my test suites could run perfectly fine on my computer but not on Bitbucket pipelines or other developers’ computers after they have cloned my repository.

Selenium is straightforward to learn and use however, the initial setup is a little challenging. This is due to the fact that Selenium requires different components (plugins, frameworks, WebDrivers, browsers, etc.) to be configured and work together. For example, to run Selenium on your computer, the WebDriver and your computer’s browser need to be of the same version. If you have Chrome browser version 88.0.4324.192 on your computer and you download ChromeDriver version 89.0.4324.192, Selenium won’t be able to communicate to your Chrome browser. Also, for Selenium to be able to find WebDrivers on your computer, WebDrivers need to be downloaded and placed into your computer’s executable path or to explicitly use setup code to tell Selenium where the WebDrivers are located. You can see how this could be cumbersome for developers working in a team and sharing the same code base.

To solve the above problem, I thought we should utilize Docker containers. Configuring Selenium to run on Docker containers will eliminate local environment dependency and make it easier to deploy and run test suites on Bitbucket pipelines since Bitbucket pipelines use Docker as well.

Docker is an open-source technology that enables developers to easily pack, ship, and run any application as a lightweight, portable, self-sufficient container, that can run virtually anywhere. Docker Hub is the platform where organizations and individuals push Docker images that developers could download/pull and use in their applications. Selenium has official images in the Docker hub that could be used for different scenarios. To learn more about different selenium-docker images and their capabilities check out selenium-docker GitHub here.

Now that we know what Selenium and Docker are, let us discuss how to easily configure Selenium so that it runs on your computer, Docker, and Bitbucket. Since we are using Python bindings, I recommend using the Pytest framework for testing because of its flexibility, easier to use, and scalability. Another cool plugin I recommend is pytest-selenium. Pytest-selenium is very useful because it takes care of WebDriver and browser configurations. It eliminates all the initial hurdles of WebDriver configurations, and also it is flexible that you can use Pytest fixtures to overwrite its configurations. Most importantly, the plugin can launch a browser locally or remotely depending on the setup in the conftest.py file.

Steps:

1. Make sure you have the following packages and plugins installed on your computer.

• Python and pip
• Docker and docker-compose
• Create a virtual environment and use pip to install Selenium, Pytest, and pytest-selenium
• Download the latest ChromeDriver or GeckoDriver depending on your browser.

Up to this stage, if you have tests, you should be able to run Selenium perfectly fine on your computer because pytest-selenium is taking care of the WebDriver configurations. ie. pytest --driver chrome --driver-path <\path\to\driver.exe> example_test.py

2. Create conftest.py file — a Pytest configuration file that uses python-selenium plugin and Pytest fixture to launch Chrome and Firefox browsers in headless mode as shown in the example below. We need to launch browsers in the headless mode because Docker cannot open browsers in graphic mode, also headless mode is faster for running web tests.

# conftest.py
import pytest
@pytest.fixture
def firefox_options(firefox_options):
firefox_options.add_argument("-headless")
return firefox_options
@pytest.fixture
def chrome_options(chrome_options):
chrome_options.add_argument("-headless")
return chrome_options
@pytest.fixture
def selenium(selenium):
selenium.set_window_size(1024, 600)
selenium.maximize_window()
return selenium

3. Create docker-compose.yml file — a YAML file that downloads selenium-docker images for Chrome and Firefox from the Docker hub. Note that the images are node and not standalone — node images are used to launch Selenium tests in the grid. Learn more about the Selenium grid here

# docker-compose.yml
version: "3"
services:
  selenium-hub:
    image: selenium/hub:latest
    container_name: selenium-hub
    ports:
    - "4444:4444"
 chrome:
   image: selenium/node-chrome:latest
   volumes:
    - /dev/shm:/dev/shm
   depends_on:
    - selenium-hub
   environment:
    - HUB_HOST=selenium-hub
    - HUB_PORT=4444
 firefox:
   image: selenium/node-firefox:latest
   volumes:
    - /dev/shm:/dev/shm
   depends_on:
    - selenium-hub
   environment:
    - HUB_HOST=selenium-hub
    - HUB_PORT=4444

4. Create bitbucket-pipelines.yml file — a YAML file that has steps to run Selenium tests on Bitbucket pipelines by using Pytest. The pipelines will run in parallel on both Chrome and Firefox browsers but you can edit the steps depending on your scenario. Note that Selenium images here are standalone because we do not have to launch the grid in pipelines since we are running both browsers in parallel.

# bitbucket-pipelines.yml
pipelines:
default:
  run-website-test-suite:
   - parallel:
     - step:
       name: run-tests-in-chrome
       image: selenium/standalone-chrome:latest
       caches:
        - pip
       script:
        - sudo apt-get update -qqy
        - sudo apt-get install -y python3-pip
        - pip3 install -r requirements.txt
        - python3 -m pytest  --driver chrome example_test.py
     - step:
       name: run-tests-in-firefox
       image: selenium/standalone-firefox:latest
       caches:
       - pip
       script:
       - sudo apt-get update -qqy
       - sudo apt-get install -y python3-pip
       - pip3 install -r requirements.txt
       - python3 -m pytest  --driver firefox  example_test.py

Another important note is that to successfully run tests on Bitbucket, you will need to have an active Bitbucket account and use version control such asgit to push your code into Bitbucket. Follow the link below to learn more about git and Bitbucket setup if you are not familiar with them already.

5. Use pip to create a requirements.txt file by issuing the following command: pip freeze > requirements.txt

6. Create a sample web test python file to test your configuration.

# example_test.py
def test_example(selenium):
    selenium.get('http://www.google.com')
    assert "Google" in selenium.title 

To run tests locally (on your computer):

Make sure you have GeckoDriver or ChromeDriver downloaded then issue the following command from your terminal: pytest --driver <browser> --driver-path \path\to\driver.exe example_test.py

Note that if the WebDriver (GeckoDriver or ChromeDriver) is in the execution path, there is no need to include --driver-path argument. Just use: pytest --driver chrome example_test.py

To run tests on Docker:

• Make sure Docker daemon is up and running.
• Execute docker-compose command to launch Docker grid:docker-compose up
• Verify Selenium grid is up and running in your localhost by navigating to the link:http://localhost:4444/grid/console
• Run tests by issuing the following command from your terminal: pytest ---driver Remote --capability browserName chrome example_test.py

To tun tests on Bitbucket:

• Make sure you have a Bitbucket account and you are logged in.
• Use git to push your code into Bitbucket, pipelines will be triggered to start running your tests.

Conclusion:

The steps above assumed you have some knowledge of Selenium, Python, and Docker. These steps are suitable for starting a new selenium project quickly and be up and running within few minutes. You can add more Selenium features like generating test reports, parallel executions, etc., depending on your project complexity.