Selenium: Growing Muscles

It has been a long time since our previous article about efficient Selenium infrastructure. If you are at the beginning of solving the challenging Selenium puzzle — please consider our posts about Selenium clusters (part I, part II), Selenoid as browser automation Swiss-army knife (one, two), Selenium on Windows platform (link). If you prefer motivational videos — take a look at my talk about scalable Selenium at SeleniumConf Berlin 2017.

Do you appreciate my Photoshop skills? ;)

A lot of things have happened in our community since the last article. Let me show you the most important features added to our tools during the last months.

Ggr

Our stateless Selenium load balancer got some cool improvements.

  1. First of all it now has an API to determine upstream hub host by session ID:
$ curl -s http://test:test-password@my-ggr-host.example.com:4444/host/8e82f31f408b4a906f715228b51...
{"Name":"my-hub-1.example.com", "Port":4444, "Count":5, "Username":"", "Password":""}

2. Also guest quota support was added. Such guest quota can be accessed without specifying user name and password. This is very useful to maintain some sandbox where users can launch their tests without any modification in their code. Guest quota is enabled using Ggr flags:

$ ./ggr -guests-allowed -guest-quota test <the-rest-of-the-flags...>

With these flags specified any browsers declared in test.xml file will be accessible without password.

3. The third cool feature is an ability to proxy Selenium sessions to external services like SauceLabs, BrowserStack or TestingBot. Some testing scenarios such as testing in mobile browsers or testing mobile applications on devices are complicated from infrastructure perspective. You can decide to maintain most of the browser infrastructure yourself and delegate any infrastructure research on these complicated platforms to commercial services. To proxy sessions to such services just add username and password to quota XML file:

<qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
<browser name="some-mobile-browser" defaultVersion="45.0">
<version number="45.0">
<region name="1">
<host name="ondemand.saucelabs.com" port="4444" count="1" username="test-user" password="my-password"/>
</region>
</version>
</browser>
</qa:browsers>

4. All our tools including Ggr are now compiled with Golang 1.9.x branch. This branch delivers even more efficient garbage collection algorithms which dramatically improve response time. Ggr is now lightning fast.

5. The most important change is VNC traffic proxying support. If servers behind Ggr have VNC daemon listening on some port — you can configure Ggr to give access to VNC traffic of any running session by its ID. Without any modification in quota files you can now access the following URL:

ws://test:test-password@my-ggr-host.example.com:4444/vnc/8e82f31f408b4a906f715228b5176efb0528ce32-66db-417f-8fa7-b12d85ce1ab5

Notice ws:// protocol meaning WebSockets. Using any VNC client supporting web sockets (e.g. noVNC) you can see browser screen of desired session. By default Ggr expects VNC to be running on standard VNC port 5900 but this if configurable.

Selenoid

The most important changes were done in our flagship open-source tool — Selenoid which is a Selenium hub successor running browsers within Docker containers.

More Configuration Options

You now have a lot more control over browser runtime environment. For example in configuration file you can pass custom environment variables, /etc/hosts and shmSize to browser container operating system:

{
"firefox": {
"default": "46.0",
"versions": {
"46.0": {
"image": "selenoid/firefox:46.0",
"port": "4444",
"env" : ["TZ=Europe/Moscow"],
"hosts" : ["example.com:192.168.0.1"],
"shmSize" : 268435456,
}
}
}
}

This allows for example to override specific browser version time zone and add hostnames from your internal network. In addition to these options you can control some settings individually for every session via capabilities.

New Capabilities

  1. If you run several tests in parallel — in order to distinct their sessions in Selenoid UI, you can specify a free-form test name using name capability:
name: "MyCoolTestName"

This is how it looks like in the UI:

Finding test session in Selenoid UI is now a trivial task

2. If you are running your application in Docker container — you can automatically link browser container to application container by specifying its name in capability:

applicationContainers: my-application-container

3. Sometimes you have to override /etc/hosts entries for only one test. This can now be done via:

hostsEntries: "some-host:192.168.0.1"

4. Finally to override browser time zone for concrete session — just specify:

timeZone: "Europe/Moscow"

Logging Improvements

We added two major improvements to logging domain:

  1. If browsers are being run in containers — any logs sent to centralized logs storage (such as Amazon CloudWatch or Google Cloud logging) can now be tagged with custom tag. This tag is specified via name capability mentioned in previous section.
  2. If you are running Selenoid without Docker — then driver process logs can be added to the main Selenoid log via -capture-driver-logs flag:
./selenoid -conf ~/.aerokube/selenoid/browsers.json -capture-driver-logs

Browser Images

We have rebuilt all browser images to support all UTF-8 locales and have installed additional fonts to correctly display pages in national alphabets such as:

  • Chinese
  • Japanese
  • Korean
  • Hindi
  • Thai

With these modifications our images can now be used to easily test multilingual websites.

Selenoid UI

We have completely reworked the main layout of Selenoid UI — a standalone user interface for Selenoid.

Selenoid UI main screen now looks like this

The most commonly used features such as browser usage statistics and browser session buttons are now present on the main UI page. Less frequently used capabilities helper widget has moved to a separate tab.

Side-by-side view

Browser screen and Selenium logs are now shown side-by-side on the same screen.

Configuration Manager

Configuration Manager — a one-command installer for our tools became even more mature. It now supports configuring Selenoid to work with Microsoft Edge and Safari browsers. CM now works well under Windows 10 and you can specify custom listen port for Selenoid and Selenoid UI e.g. to simultaneously run Selenium server and Selenoid.

Configuration manager colored output

We have also made CM logs more human readable by adding colors to the output.

Video Recording

The cherry on the cake is the recently added video recording feature. In addition to looking at real-time browser screen in Selenoid UI you can now capture browser screen and save it to H264 video file. A typical video file can look like this:

To start capturing the video simply add one more capability to your tests:

enableVideo: true

By default all video files are saved as <sessionID>.mp4 where <sessionID> is a unique identifier of the browser session that can be logged in tests. If you wish to provide custom name - specify it as videoName capability:

videoName: "my-cool-video.mp4"

In addition Selenoid automatically serves folder with video files via HTTP. To open video file in browser — go to the following URL:

http://my-selenoid-host.example.com:4444/video/my-cool-video.mp4

To open the entire list of available video files just omit file name:

http://my-selenoid-host.example.com:4444/video/

I hope you now have a lot more motivation to deploy really efficient Selenium infrastructure. Trust me — browser automation can be painless. If you have any questions — don’t hesitate to write us by email, to Telegram support chat or by posting a question to StackOverflow tag.

If you are already using Kubernetes you can be also interested in our new project — Moon which is a native Selenium solution for this platform using all best features of Selenoid.

À la prochaine.