Investigating a Mobile Usability Lab


We’re in the early days of establishing our own in-house usability testing lab. This will enable us to rapidly test prototypes and design concepts in the convenience of our offices and is a critical part of our user experience design process. There’s nothing more illuminating as seeing your ideas in the hands of your audience.

Sure, it would be great to have a lab with a separate viewing room (like this) or a set-up where we could test all possible platforms and devices but, in the spirit of starting small, we knew we had to keep it light, fast and cheap. We decided to begin by outlining our key requirements to help focus our investigation.

Considerations for mobile testing

Testing for mobile has its own set of challenges that differ from desktop. How might we record a participants’ reactions? How can we obtain a clear view where hands aren’t obscuring the area of the screen the participant is interacting with? How and where can we record the types of interactions they use, such as long-tap, swipe? How can we ensure a quality recording that can be used to document sessions, extract insights and be presented to the client? And of course, how can we ensure that the set-up provides the ability to test the multiplicity of device screen sizes, resolutions and platforms that are on the market?

We also knew that we would be building a html prototype due to the fact that we had some experiments in UI we wanted to test that a simple tap-through prototype tool such as Pop or even Axure would not be custom enough to display faithfully.

Naturally, we turned to the great sea of information around this topic online, yet we quickly discovered that budget, timelines and our specific requirements would certainly shape what we would be able to implement as our first set up.

Using our requirements as a guide

To help focus our investigation, we began by outlining our key requirements and, consequently, we felt that any software and hardware solution would have to be:

  • Unobtrusive for participant and facilitator by keeping any device attachments, webcams, lights, external monitors to a minimum
  • Low training overhead so that any team member within the business could use the set-up for their project with minimal training
  • Cost effective — without the need for expensive software or hardware (remember, we’re aiming to start small here!)
  • Able to record facial expressions and screen-view within the same output video so there’s no post production required
  • Software that is touch-point enabled (this is usually represented with a small animation at the co-ordinates where the user touches the screen)
  • Flexible and able to record both browser or app-based prototypes and ideally platform agnostic.

The three approaches

With these requirements in mind, we looked into three potential approaches for a set-up.

1. In-phone recording app

There are new apps coming out all the time that incorporate screen recording and video capture through the onboard camera to offer a truly ‘mobile’ testing lab. All you need to do is install the app on a phone and off you go. No wires or external webcams needed. We tried a number of these such as Magitest and UX Recorder. They usually make use of the phone’s own front camera and mic to capture participant’s facial responses during the session, while screen recording runs in the background to capture screen activity through their bespoke browser environment. With the idea that it is the closest to a natural browsing experience and be unobtrusive. Sounds perfect right?

Benefits:

  • Cheap to buy (often free or pay-per-session)
  • Simple to set up (download, install and type in your URL)
  • Uses the phone’s native software and hardware so it’s very unobtrusive
  • Can use on the street giving us freedom from the lab.

Drawbacks:

  • During our testing, using all those in-phone functions was deadly for the processor, making it a very laggy experience
  • Not all software packages are equipped to to record the user’s touch-points
  • A risk of filling up the phone’s storage if recording 1 hour long sessions
  • Have to use their browser environment — not able to test apps.

When would this be useful?

This would be an ideal set-up if you wanted to head out into the streets to conduct guerilla usability testing as all you need is your phone. Or perhaps if you have a good spec of phone with plenty of processor power and storage space.

A fit with our requirements?

It sounded great on paper, however, we found the processor burden of running the app’s browser and the phone camera and mic concurrently created a very choppy experience that unfortunately made this option simply unworkable.

2. Mounted camera rig

We next researched the use of mobile phone rigs as the idea of freeing the participant from patiently holding their phone under a webcam (as experienced in many lab set ups) was enticing. These rigs are usually metal or plastic devices that clip onto a phone and have an arm that stretches over the top that you attach a webcam to. You can go the professional route with a device like Mr Tappy or a DIY approach.

Benefits:

  • Lightweight attachments
  • Freedom for participant to sit more comfortably on, say, a couch rather than sat at a desk
  • Can observe how people are holding devices and the stretch required to access all areas of the screen
  • Platform and device independent as you can clip it to anything.

Drawbacks:

  • Can be pricey to purchase, or time consuming to construct DIY version
  • Single orientation; you would need to re-mount the camera if the participant changed orientation
  • The rig may need to be adapted to accommodate a second camera to record the face
  • The participants hands can obscure the view of the screen they are interacting with
  • The rig attachment makes it awkward to hold the device.

A fit with our requirements?

As swanky as Mr Tappy would look, we did not have the time or budget to invest in testing this approach and quickly realised we would still need to address how to get an image of the facial reactions as this is a very important part of the unspoken user feedback.

For this rig to be suitable for our requirements we would also need to work out how to capture the face, a clear view of the screen (unobstructed by the participants hands) and record the touch-points. All absolutely possible through using additional webcams and the correct software.

3. Silverback + Reflector + custom JavaScript

Testing the setup. The laptop’s built in camera records the participants face, while javascript is showing the touchpoints on the mobile and the phone’s screen is mirrored to the laptop via Reflector and…
… the whole scenario is recorded on the laptop screen by Silverback.

After concluding that the mobile rig and in-phone solutions were unsuitable for us at this time, we explored how we might utilise a piece of software called Silverback to extend it to mobile testing. Silverback is a desktop usability testing tool for mac that captures the desktop screen along with a video of the user via the Facetime camera and audio via the internal mic. Aside from how it captures the sessions, it also boasts a host of features for more rapid post-production, such as adding highlight timestamps to the video timeline while in session.

Making the connection between desktop software and the mobile prototype

By using Reflector to mirror content from the iPhone wirelessly to the Macbook, we were then able to capture all of this onscreen with Silverback. We were able to record the participant’s facial reactions and see what they were looking at on the mobile as well as their screen, but what about where they’re interacting with the screen? For this we enlisted the help of our development team who wrote us some custom JavaScript that we applied to our HTML prototype.

When the screen is tapped, it fires an event to discover the X & Y coordinates of the tap. It then displays a mark on the screen at that site.

And there you have it — a quick and dirty gesture indicator. You will need to add this line of custom javascript to your website to enable the touch and trigger point — so this works well if you are coding a prototype in html to be viewed in the browser.

Benefits:

  • Silverback and Reflector are cheap and cheerful pieces of software that are also easy to use
  • Silverback can be used for both desktop and mobile testing scenarios
  • Can capture anything that is on the phone’s screen, so free to test through both browser and apps
  • Reflector can now wirelessly mirror both android and apple phones.

Drawbacks:

  • Silverback is only available on iOS (however, you can still mirror
    android to it)
  • Using the laptop FaceTime camera to record the participant’s face means that it’s harder to see how they are holding the device
  • You will need to use prototyping software that is touch point enabled (such as proto.io) or add custom JavaScript to your own prototype as we have done.

When would this be useful?

This set-up makes good use of desktop software by repurposing it to facilitate mobile testing which is great. It’s a quick workaround that can be used to set up a lab.

A fit with our requirements?

We used this set-up successfully in a number of usability sessions. Silverback proves to be a great multi-use software option and Reflector can also be used for this and other purposes around the office, such as presentations. It doesn’t take much time at all to get Silverback, Reflector and the JavaScript all set-up and working, which makes it a pretty good system for what we need it for.

Summary

It’s a bit of a minefield getting to the bottom of what software and hardware will work for you in your specific scenario but, with patience and a bit of tinkering, it looks like you don’t need to spend a packet to record users and their reactions to your products.

Our next step is to get this system working for testing android devices. Perhaps even some eye-tracking options could be thrown into the mix.

Have you created a similar set up in your studio? What cheeky workarounds did you discover to fulfil all of your mobile testing requirements? Let us know, we’d love to hear from you.


Referenced apps & software

Magitest— Simple mobile usability testing for sites and apps. Everything is captured and recorded on the device with no additional hardware required.

UXRecorder — Mobile web site testing for iOS.

Mr Tappy — A kit for filming mobile phones, tablet computers and handheld devices from the user’s point of view.

Mobile testing sled — Make your own iPhone usability testing sled for £5.

Silverback — Version 3 now available. Guerrilla usability testing on the Mac made easy.

Reflector — Reflector is a wireless mirroring and streaming receiver that works great with Google Cast™, AirPlay™ andAirParrot.

Proto.io — Create fully-interactive high-fidelity prototypes that look and work exactly like your app should. No coding required.