How to set up cross browser testing services

In the first post about cross browser testing on the Bitrise Blog we gave you a detailed comparison of their features, so now in the second blog post, let’s see short tutorials for setting them up.

Goals

What do we want to use it for?

• UX change validations.
• Integrated test sessions for UI.
• Running some local tests.
• Running some remote tests on a CI server.

What do we plan for the future?

• Fully automated tests on every design change.
• Integrating tests with GitHub/BitBucket and with Slack.
• Easy access or screenshots for any failed test (comparing screenshots or watching the videos).
• Archiving tests after a 2–3 day period.

What is the meaning of the life?

.

0, Before you start

First, we will setup the local test run. For this, we will need to set up an app for testing. In my example, I use an Angular Universal PWA with the Angular CLI, which has already generated a protractor.conf.js for the end-to-end test.

What we need to do now is just simply run

$ ng e2e

After a successful run, we can set up some custom assertions and expectations. I recommend using Page Objects to make it easier to do the same tests on lots of pages. You can read more about how to write it here.

So when we have our test ready and it is running locally, let’s create a local connection with Sauce Labs.

1, SauceLabs

To get a free account, register. After you’ve logged in, you will see the dashboard.

Run a local connection

First, you’ll need to download a small binary from here. After you’ve downloaded the right version, go to the appropriate folder, for example, ./sc-4.4.12-osx

$ ./bin/sc -u $SAUCELABS_USERNAME -k $SAUCELABS_AUTHKEY --tunnel-identifier myTunnel --no-remove-colliding-tunnels

• Simply add your local environment variables.
• SAUCELABS_USERNAME is NOT your email address
• SAUCELABS_AUTHKEY can be found by selecting User Settings in the top right dropdown and scrolling down to the Access Key section. (Click Show.)

This is what you’ll see once you’ve run the code above:

If you have successfully set up the connection, you will see something like:

# Sauce Connect is up, you may start your tests.

And in the Sauce Labs console, you should see that a tunnel called “myTunnel” was created and is active.

Setup and run Local test

In your protractor config you’ll need to add some code, like this one:

exports.config = {
    sauceUser: process.env.SAUCE_USERNAME,
    sauceKey: process.env.SAUCE_ACCESS_KEY,
    // sauceAgent: sauceRestAgent,

There is a good configurator available to create code snippets for the desired capabilities of your automated testing scripts in the language of your choice, quickly and easily.

• Update protractor.conf.js with sauceUser and sauceKey
• Add tunnelIdentifier: ‘myTunnel’, to the capabilities
• Run Local Proxy connection
• And in an other terminal window run the e2e test with the following command:

$ ng e2e

Annotate your test

Without annotating your test, it will show only a complete status, so it's worth doing it.

You can add a Failed or Successful status to your test, see a detailed tutorial here. It can look like this after configuring it:

In addition, you can add a context (describing steps so that you can see at a glance what is what) to your test step for readability:

// spec.js
describe('Protractor Demo App1', function() {
  it('should greet the named user', function() {
    browser.executeScript("sauce:context=Going to 'angularjs.org'");
    browser.get('http://www.angularjs.org');
    browser.executeScript("sauce:context=Sending text to name input field");
    element(by.model('yourName')).sendKeys('Example');
    browser.executeScript("sauce:context=Asserting 'Hello Example!' text is present");
    var greeting = element(by.binding('yourName'));
    expect(greeting.getText()).toEqual('Hello Example!');
  });
});

It’ll look like this on the UI:

We can add browser.executeScript("sauce:job-result=passed"); when all the tests have passed using Jasmine's onComplete function or we can add browser.executeScript("sauce:job-result=failed"); if any has failed.

Also, you can implement SauceLabs = require("saucelabs"); and call the updateJob function and update by sessionId. SauceLabs does not do these automatically for you.

Add a ‘success job’ to your code

saucelabs.updateJob(driver.sessionID, {
      name: title,
      passed: passed
    }, done);

See a complete (working) code here: https://gist.github.com/dbreuer/30c0d0d0f47d255fe0613f72d14322b5

2, Crossbrowsertesting.com

Get a free account and register here. After you’ve logged in, you will see the test center.

Run a local connection

There are two ways to do it:

1. When you install the chrome app, just click the local connection button on the top navigation and install the app. Then refresh the page.
2. Click the local connection button when it needs to be connected. In our case, I recommend the second solution.

So you could use the node package:

$ npm install -g cbt_tunnels
$ cbt_tunnels --username $CBT_USERNAME --authkey $CBT_AUTHKEY

You will find the username and access key under Account settings => Manage Account.

You’ll now see this after running the code above:

Setup and run the Local e2e test

We already have our app set up so we just need to change some of our code. (If you need more help with setting it up, check this repo’s readme.)

Check out the configurator in the testing center under Automation => Getting started page or you can find the complete code here.

1. Update protractor.conf.js
2. Run Local connection (either chrome app or cbt_tunnel package)
3. Run the e2e test:

$ ng e2e

DO NOT forget your http://localhost will be http://local.

Annotate test passed or failed

To do that you need to add some functions and call them, like in the following example. You can copy this code into your config file.

3, BrowserStack

Register a free account. After you’ve logged in, you will see the Automate page.

Run local connection

Visit the Local testing page and download the binary for your local connection. Go to your folder into which you’ve downloaded the app and you’ll have to run the following code:

$ ./BrowserStackLocal --key $BROWSERSTACK_AUTHKEY --local-identifier Test123

You will find the BROWSERSTACK_AUTHKEY in the console left side under the “Username and Access key”. Just click “Show” and copy paste the access key.

Setup and run the Local test

You’ll find this guide about Protractor Integration with BrowserStack helpful or check out a piece of working code. Then:

1. Update protractor.conf.js
2. Add ‘browserstack.local’: true, to the config file
3. Add ‘browserstack.debug’: ‘true’ to the config file
4. Run Local connection
5. Then Run a test with $ ng e2e in another terminal window.

Annotate test Passed or Failed

For setting up annotations, check this guide.

The BrowserStack test results page compiles all the data you’ll need:

You need to create a function to update the session via the REST API by the browser sessionID. I’ve come up with a solution but it is not clean and safe, so keep looking to find the best for this, until the official documentation suggests otherwise.

Which one(s) of the testing services do you use? Share your thoughts with us!

Originally published on the Bitrise Blog.