Web Application Testing

Modern Approaches and Techniques

8 min readJan 13, 2014

Introduction

Today, users can access an application from any platform regardless of their browser, operating system, configuration, resolution, or viewing device. This means that device and resolution-independent testing should be a continual part of the development cycle from the very start of a project in order to address issues as soon as they emerge. Given the fragmentation of technologies that users could potentially be utilizing, this can seem like an onerous task. In the past, this would require physical devices to access the application, one-by-one, in every feasible environment. While real devices should still be part of your toolset, emulation and visualization offer convenient testing services to augment these approaches.

As such, traditional and preconceived ideas of testing must be revised to fit with modern technologies. Testing can be done in lots of different ways, but here is a proposed three-pronged simultaneous approach:

Validation/Decision-Making with Real Data
Ongoing Testing with Physical Devices Locally
Iterative-Based Virtualization/Emulative Testing for Browsers

Data-Driven Support

Undoubtedly, decisions are made (and rightly so) as to what platforms are chosen to be supported. What platforms are your users using currently? What platforms could they be using in the future? These decisions should be based on analytical data from actual users through technologies such as Google Analytics or other alternatives.

These metrics are critical to the decision making process and directly impact the timelines and required resources of application development. It also greatly informs developers on what should be prioritized and what technologies can and should be used. This is one piece of a methodology called Data-Driven Design, where actual data is used to help make decisions [Further reading: http://uxdesign.smashingmagazine.com/2013/09/24/data-driven-design-in-the-real-world/].

Other benefits to this approach is that it :

greatly mitigates incorrect assumptions
eases politically-charged decisions by taking from personal ownership
educates/inspires application builders to find better solutions

This approach should be constant through the development process whether it’s application goals and requirements are being set, design decisions during development, collecting ongoing user feedback through prototyping, or improvements are being considered for future iterations.

Ongoing Testing

Creating an application in a vacuum doesn’t help anyone. When creating solutions used by an enumerable amount of users, one size won’t fit all. That means that a mobile-first approach is required in order to serve the needs of user access. When your application isn’t available (either functionally or useably) then an opportunity is missed — one your competition is happy to take advantage of. One foundational approach is by testing locally across multiple devices simultaneously so that every css/html/javascript change can immediately be seen across multiple devices.

There are a few tools out there that provide constant refreshes and even remote control across devices, especially while developing locally.

Displaying Realtime Development Locally with Adobe Edge Inspect

Ongoing Dev Tools: Adobe Edge Inspect & Ghostlab

The two major players in this space are Adobe Edge Inspect and Ghostlab. Both products operate in similar ways, but have a couple features that make them complimentary to one another.

Both use a wifi connection to share content across multiple devices. You hook all your mobile devices up to the same wifi network, use your laptop or desktop to serve the site, then watch as each device renders the application. Andrew Jones has a great article pitting the two against each other in the reference above. Here is his quick comparison:

Edge Inspect: The Good

Fast, (mostly) accurate rendering across devices
You can take screenshots on each device (although I’ve never actually used this feature)
While the app is running on a device, the screen does not go to sleep. This is a nice feature, as you’re not constantly clicking and sliding on devices to wake them up.
FREE (with a paid Creative Cloud member account). A pro or con, depending on which side of the Creative Cloud subscription debate you’re on. I personally love it.

Edge Inspect: The Bad

Requires installing and running a standalone app to do any testing
Requires Edge Inspect app to be installed on every device.
Requires installation of Chrome plugin for managing devices, debugging, manual reloads, etc
Currently can only be run on Android and iOS devices. Meaning, no testing on other browsers, platforms, Microsoft Surface, etc.
Often has a real problem loading external Typekit fonts. On any given day, at least one device will refuse to load web fonts from Typekit. Normally it’s the iPad, but it’s a crapshoot and completely random.
Sites aren’t loaded directly in the device’s native browser itself. I’m assuming on the iPad/iPhone, it still uses Safari, but I worry about it being accurate sometimes.
Sometimes devices will just “forget” to refresh and you have to manually hit the refresh button from the Chrome plugin
You can’t tell Edge to just mirror a single tab in Chrome. Every single time you select a new tab, that tab’s URL is loaded into each device. This is super annoying when you’re needing to flip back and forth to Google, Stack Overflow, or just check in on Facebook. You know, in case anything important is going on
Requires a paid Creative Cloud membership

Ghostlab: The Good

Can be used in any browser on any platform, no additional software required. I tested no less than 9 borrowed devices today and didn’t have to make them install an app on every one.
Setting up multiple entries for multiple test sites saves time.
You can specify the debugger you want to use (haven’t tried this, though)
So far, I’ve had zero problems with external web fonts loading
Synchronized scrolling cuts down on swiping fatigue on multiple devices
Works on both Mac and Windows

Ghostlab: The Bad

Inexplicable phantom scrolling from time to time
Mouse clicks on javascript events don’t fire the same event on other browsers. Not even sure this is possible, just throwing that out there.
Devices constantly going asleep is annoying. I realize a solution like Ghostlab can’t possibly fix this, but it’s a pain nonetheless. I’ve resorted to telling my devices to never go to sleep, which can be a battery drain, especially if I walk away and forget to turn them off.

Each one has some overlap, but I can see a real value in using Ghostlab for local development then using the extra features of Inspect to take screenshots for documentation/sharing with the team and to present to shareholders once it’s hosted externally. Since most UI designers are already using the Adobe Creative Suite, Inspect is a freebie, but Ghostlab makes a compelling product with the extras it offers over Inspect.

While mobile devices are great to have on hand, you also need to test across multiple desktop browsers, especially the ancient ones folks still use.

Iterative-Based Virtual/Emulative Testing

While developing for mobile should be your first line of testing, they typically render content in the same way using Webkit, so the rending engine is usually the same. This lets you use the latest and greatest coding practices and will consistently render across all those browsers very similarly. The only differences are that there are multiple ways to render content based on the device being used. But what about desktop users who are using different browsers, often browsers that don’t conform to the Webkit standard. What about supporting the dreaded Internet Explorer?

While globally, it’s share is now less than 30% worldwide (as of Nov. 2013 — reference: http://gs.statcounter.com /#browser-ww-monthly-201211-201311) it’s as high as 40% in the U.S. and even higher, depending on your target demographic.

This is where virtualization steps in: instead of having to have several physical computers running every OS/Browser combination you want to support, testing between every single one, you can use services like BrowserStack or Sauce Labs to use their server farms to render your site across tons of platform combinations to see how well it’s showing up. They also offer Selenium Testing which offers automation of various tasks, like clicking all navigation links to see if they work or submitting forms. This greatly reduces the headaches of having to build your own browser testing infrastructure or manually test interactions.

Subscriptions to these services also allow you to test your application as many times and as often as you like. This is a perfect way to push applications to an external server periodically, then run your battery of tests to make sure you’re golden.

Combined with the mobile-first device testing, this keeps your development airtight throughout the entire process, saving you the many headaches that would come up if you found out after everything was done.

In-Browser Viewport Resizing

While physically testing the application’s presentation on multiple devices is a critical step, much of the responsive flow of the layout can be tested by simply resizing a desktop browser. An easier, relatively accurate way to test this is by using a plugin called Resizer, from Malte Wassermann.

Resizer Browser Plugin

This is a great plugin that changes your viewport without having to grab the edge of your browser and manually change its size. It also offers many common device resolutions so that it can be easy to see how the presentation will flow across them.

Final Thoughts

By continually testing across multiple devices, browsers, and operating systems in real-time alongside development, you can fix problems immediately as they show up. Add in frameworks like Twitter Bootstrap, and a large part of your cross-compatibility foundations already work in the majority of browsers as well, saving you from tons of hours of troubleshooting right out of the box.

These technologies allow developers to work faster, less-expensively, and with a higher degree of quality all while getting constant feedback of progress along the way. It’s a great time to be creating for the Web.