Running Automated Tests on AWS Device Farm in Custom Environment
There are some certain drawbacks of running your automated in tests in AWS Device Farm standard environment especially when your tests are dependent on a desired execution of order. Such a sequence is not followed in standard environment. What’s more, the test execution is slow in standard environment as compared to custom environment due to granular reporting in standard environment.
There are several cloud testing providers out there such as Saucelabs, Browserstack, Testingbot and many others. I opted for AWS Device Farm because we were already extensively using AWS services, so decided to stay on the same stack. I’m of the opinion that AWS Device Farm isn’t the perfect solution yet, it doesn’t work out of the box (Such as changing hub URL and Desired Capabilities), but it has a potential for being a solid cloud testing solution since it is backed by Amazon
Now let’s move to our main topic
AWS Device Farm Setup
Firstly, you need to signup on AWS Device Farm and after successful signup they will give 1000 free minutes to get started.
Automated Tests Packages Setup
Usually, on some cloud testing provider such as SauceLabs, the test code will run on your computer while instructing cloud devices to do something, but in Device Farm you need to package all your tests to be run on the cloud.
Here are things that I used for my testing framework:
- Appium
- Java
- TestNG
- Maven
a. Create & Configure testng.xml
Create testng.xml file under src > test > resources > testng.xml
For example my testng.xml file looks something like this:
Add this plugin to your POM.XML
b. Packing the tests
Use maven JAR plugin to pack tests into single jar file and send to AWS cloud server.
Add this to your pom.xml file header under version tag.
<packaging>jar</packaging>
Add this plugin to your POM.XML
c. Packing Dependencies
Now we need to pack all test dependencies into a single jar file as well using Maven dependency plugin.
Add this plugin to your POM.xml
When we compile the code, we should see two JAR files under target directory. One containing your target tests and other containing dependencies
d. ZIP Everything (tests and dependencies jar files)
Create a new file src/main/assembly/zip.xml and put this inside.
And put this into your POM.xml
e. Package Everything!
Go to terminal and type
mvn clean package -DskipTests=true
You should see the following upon successful packaging
At this point you should have zip-with-dependencies.zip file under target directory. Which will be uploaded to aws device farm later on..
Run Tests on AWS Device Farm in Custom environment through testng.xml
- Login to AWS Device Farm
- Create a project
3. Create a new run
4. Upload APK file which is under test
5. On configure a test page, select appium java testng and upload zip-with-dependencies.zip file
Select Run your test in a custom environment radio button
6. Configure YAML file to run test in custom environment through testng.xml
Click on Edit button and configure the custom environment majorly for appium version and to control the tests through testng.xml. Given below is yaml file that I’ve configured to run my automated android tests for Pliro Mobile app. You can copy paste as it is and save the nameyourYAMLfile .yaml file
7. Select Device pool or you can use the default one
You can create your own device pool depending the number of devices required. Tests will run faster if you opt for a single device pool. Just simply click on Create a new device pool button and select the device.
8. Specify device state
just leave this page as it is and move onto next page
9. Review and Start Run
10. Grab a cup of tea and wait for results :-)
11. Check the results
Wow!!….Congratulations, all your tests passed!
Conclusion:
About the Device Farm, in my short time I haven’t been able to find much on the below issues:
- Logging. There are some logs but so far I haven’t been able to get the log of my test. Just appium & device logs.
- Geographic Location. You are going to have issues while running tests specific to a Geographic Location. Location change is disabled on device.
- Slowness. Test execution is slow when we add multiple devices
On my next blog, I will show you how to automate your test execution on AWS Device Farm using a Jenkin’s pipeline.
