Recipe for Appium iOS App Testing

The Original Recipe

It is easy to setup automated tests with Appium. Just follow the install steps, set your preferences, and you are ready to write and run tests on the Appium server.

This basic setup works, but there is much more that can be added and tweaked to make the test suite robust and easy to use. This is especially important, if you are the only QA engineer on a team and can’t devote all your time to maintaining the test suite.

Adding To The Recipe

Just like with baking or cooking, adding the right additional ingredients to your automated test suite can create a better end result. Here are some suggestions that we have implemented at Paperless Post to help create a more flexible test suite.

1. Have a standardized approach to the project folder structure, which makes it easy to find the tests you are looking for. In addition, separating test scripts by device and feature type allows QA to easily find and run any number of tests.

1. Create individual files for variables, user data, hardware, and the Appium setup. This structure allows variables to be defined and easily updated in one location without updating each individual test script. If a variable changes and causes tests to fail, this setup makes it easy to find and update that variable to get the tests passing again.
   Excerpt from the iphone_global_variables.rb file:

• ##KEYBOARD AND POPUPS
  $iphone_select_all = “//UIAEditingMenu[1]/UIAElement[@name=’Select All’]”
  $iphone_kb_delete = “//UIAKeyboard[1]/UIAKey[contains(@name,’elete’)]”
  $iphone_kb_return = “//UIAKeyboard[1]/UIAButton[@name=’Return’]”
  $iphone_kb_q = “//UIAKeyboard[1]/UIAKey[@name=’q’]”
  $iphone_kb_search = “//UIAKeyboard[1]/UIAButton[@name=’Search’]”

1. Add Command Line variables to the Appium setup file to make it easy to switch between different devices, device type, and staging environments.

• $device = ENV[‘DEVICE’] || ‘ipad_air_ios8’
  $environment = ENV[‘ENV’] || ‘Production’

1. This allows us to set test preferences on the command line without needing to edit any files.

• $ DEVICE=iphone_6_plus ENV=Staging rspec spec/iphone/login/login.rb

1. Use RSpec tags in the tests to create cross feature test suites.

• it “Successcul Create Account”, :iphone_daily_smoke_run => true, :iphone_regression_run => true do

1. Then create Rake tasks to make it easy to execute those test suites on the command line instead of a long winded RSpec tag statement.
   Without the rake task:

• DEVICE=ipad_mini_9 ENV=Staging rspec spec/ipad/**/*.rb — tag ipad_daily_smoke_run — format html — out ipad_smoke_02052015.html

1. With the rake task:

• DEVICE=ipad_mini_9 ENV=Staging rake ipad_daily_smoke_run

1. Look for alternate ways such as using API calls to verify test results, when Appium is lacking. Appium can only see what is happening on the frontend of an app, and often there are elements that are invisible to Appium. Using API calls is one way to verify that test commands resulted in the proper behavior and don’t add more than a few seconds to test times.
   One example of the above is the test to add a package background. Appium can see the buttons to add a background, but not the actual background in the display modal. Instead, we use an API call to verify that the background was added and saved to the card package.

• x = draft_expanded_details(user, pwd, draft_id)
  begin
  background_starting = x[“card”][“background”][detail]
  rescue
  background_starting = “No background present”
  end
  return background_starting

Adjust The Baking Instructions

Once your tests have been written, they are only as reliable as the server and settings on which they run. Basic server setup can cause otherwise well written tests to be brittle and flakey.

1. Build intelligent and dynamic wait statements that allow for the app and its elements to be loaded before trying to execute test steps.

• def wait_for_paper_preview_to_load
  num = 1
  while element_visible?($iphone_ordpre_loading_icon) && num < 11
  sleep 1
  num += 1
  end
  if num == 10
  puts “~~~IT TOOK LONGER THAN 10 SECONDS FOR PREVIEW TO LOAD. THIS TEST WILL FAIL~~~”
  end
  end

1. Make Appium install a fresh copy of the app for each test by using the `full-reset` option. This forces the Appium server to install a copy of the app at the beginning of each test and then uninstall it at the end, ensuring that every test starts with the same app and same basic setup.
2. Have the Appium setup file start and stop the server with every test. When the Appium server is always running in the background, it can get into a dirty state and cause your tests to fail. Starting the server fresh for every test ensures that there are no commands or error state leftover from the previous test.

Serve With A Side Of Automated Maintenance

Automation doesn’t exist just for testing. It also can be used to speed up and reduce manual effort when performing test maintenance:

1. Write scripts to create and maintain test accounts, which tend to get erased when staging environments and their databases are reset.
   Excerpt of a test account creation task:

• while num < 6
  login = RestClient.post url, {:account => { “email_address” => email[num], “password” => pwd[num] }}
  @response = JSON.parse(login)
  if “success” == @response[“status”]
  puts “SUCCESS: #{email[num]}”
  else
  puts “FAILURE: #{email[num]}”
  end
  num +=1
  end

1. Get Bash to build the latest version of your app for you and save yourself from typing multiple commands.

• echo “Reset and clean directory”
  git reset — hard
  git clean -d -f
• echo “Checkout and pull from selected branch”
  git checkout origin/master
  git pull origin master

End Result

By adding to the basic Appium recipe, you can greatly reduce manual testing for apps and build QA standards. Using these strategies, the iOS team at Paperless Post:

• Reduced the QA load for an iTunes release candidate from 5 days to 2!
• Set up an automated smoke test that runs every morning against all support devices to ensure the minimal level of functionality is maintained.
• Freed up QA resources to test new features, by automating regression testing
• Reduced downtime when variables and test flows change.