File upload in e2e tests using SauceLabs
This post explains how we upload test documents to SauceLabs server in two different ways and what is the difference between them.
I work on the team at Earnest which supports loan applications. Loan applications often require a user to upload supporting documents, for example that might be bank statements or a W2. To ensure that our system is able to accept documents, our e2e tests upload a fake document and verify that it was successfully uploaded.
At Earnest we use Selenium webdriver with NightwatchJs test framework to accomplish this.
We could run our e2e tests from a local machine by using the browser. We do that for quick local testing. But that is not scalable and doesn’t allow us to test multiple browsers. In order to scale, we use SauceLabs, a third — party service that records steps and creates a video of each test (helpful in pinpointing the cause of failure). SauceLabs also allows us to test applications on different browsers, browser versions, and different operating systems.
With SauceLabs, the tests are running on one machine; the browser is on a separate machine (managed by SauceLabs). We need to upload a file from the “browser machine’s” file system to our application. This means our tests need some way to place a test file onto the browser’s machine before testing the upload.
For uploading a file to a remote server such as SauceLabs, Selenium webdriver has a handy built-in setFileDetector method that makes placing a file on the server very easy. But it is written in Java and it does not work with NightwatchJs framework. So we came up with a different solution.
The following diagram shows how the two approaches work:
Figure 1: Architecture
Explanation
setFileDetector() method in the green (setFileDetector()) scenario places a local file directly onto SauceLabs server via WebDriver.
NightwatchJs on the other hand requires two more steps, namely configuring pre-run script and using curl to fetch a file and place it onto a SauceLabsVM
E2e test then finds the file in SauceLabs remote server and uploads it into the web app. Finally the test verifies that the correct file has been uploaded.
How NightwatchJs Approach Works
SauceLabs runs tests on virtual machines. If the browser is Chrome, Firefox, or Safari then the VM is Linux- or Mac-based. If it is IE or Edge, the VM is Windows-based. In order to test the uploaded file we first need to place it onto each virtual machine. This is done with a pre-run executable.
A pre-run executable is a script that is used to change browser settings or virtual machine configuration prior to running the tests. We use the script to fetch a file from somewhere on the Internet.
In case of Chrome, Firefox, and Safari we use Bash script with Curl command to fetch a file. This is because these browsers are on Linux or Unix operating system that supports Curl.
In case of IE we use a BAT script with BITSAdmin utility.
The set of steps is as follows. Example for IE:
- Add pre-run command to NightwatchJs configuration
prerun: {
executable: “https://gist.githubusercontent.com/username/xxxx/raw/yyyy/prerun.bat",
background: false
}
2. Place pre-run script somewhere in the Cloud for example Github Gist
3. The pre-run script uses Curl or bitsadmin to fetch a file from some internet location and downloads it onto SauceLabs virtual machine local storage at the path we specify.
@echo off
bitsadmin.exe /transfer “prerun_ie” https://gist.githubusercontent.com/user/xxxx/raw/yyyy/foo.pdf C:\bar.pdf
4. When SauceLabs runs e2e tests in that VM, it points the browser to the fetched file when using it to test for uploading
5. E2e test finds the file in the VMs storage and uploades it into the loan application.
6. After document is uploaded, the e2e test asserts that the document is present.
Potential Concerns
We need to ensure that the test image is in reliable location on the Internet. E2e test will run on deployment pipeline several times a day and if test image is suddenly removed from its location then the tests will fail. It’s best to pick a file that resides on the company’s server with carefully chosen access privileges.
We also need to be careful to use only insensitive test documents because VMs where those documents reside are shared with other parties.
The file upload can be done without built-in setFileDetector() method by using pre-run executable. The important distinction is that for Windows-based browsers and VMs, pre-run executable needs to run Curl equivalent command. Curl itself won’t work on Windows.
Resources:
