Mobile UX: The Undiscoverable Country

Act III, Scene I.

Enter HAMLET frantically typing on his phone.

To swipe, or not to swipe: that is the question.
Whether ’tis nobler in the mind to suffer
The slings and arrows of outrageous animations,
Or to take arms against a sea of confusion,
And by opposing end it.

The undiscoverable country from whose boundary
No user returns: it puzzles the will
And makes us rather bear those ills we have
Than flee to others that we know not of.

…Which one of these nondescript boxy icons is the Save button anyway?
Why does everything vanish when I scroll?
Woe is me. I’m Danish, aren’t we supposed to have good design here. Fuck.

Ok, so maybe not every user of a mobile app is driven to contemplate suicide. But it’s an unfortunate reality that many apps are maddeningly confusing and loaded with unpredictable interface behaviors.

Nobody sets out to make a frustrating app. Rather it’s a product of problems that accumulate during a development process which often manages to be both rigid and unstructured at the same time.

It all starts with an unclear core idea. Why does the app need to exist? The worst answer is “we should have one”. If the app concept starts as a bucket where all stakeholders can pour their hopes and random ideas, the end product won’t do anything for anyone. People are not looking to “interact with a brand” on their personal device. Mobile users are laser-focused on tasks: if it takes ten seconds to navigate through the app’s menus to discover functionality, the user is already lost. Giving up is always just one home button tap away…

But that’s all still in the distant future as the team blissfully collects their ideas. After the requirements have been haphazardly assembled and immediately set in stone, the project ends up on a designer’s desk. He’s really more at home with kernings, color spaces and keyframes rather than navigation controllers and app bars… But the great thing about apps is that everyone uses a phone all the time, so everyone knows how they’re supposed to work. It’s mostly a matter of making it look nice.

He recently saw an interesting blog post about how user interfaces in mobile apps should delight the user. There’s clearly nothing delightful about lists and tabs that look like the ones in the phone’s built-in apps; it’s hard to be delighted when you know exactly what to expect. Delight has an element of surprise, so let’s make sure that the user is delighted right off the bat. Instead of a plain old list, could we make these items have personality. This project turns out to be a great opportunity to design an interaction that the world has never seen before. Time to hit After Effects!

At client review, everybody loves the animation. There’s also a PDF filled with wireframes, but they are gray and boring and nobody really looks at them. What’s there to see anyway? That stuff is kind of obvious since everyone can use a phone.

Eventually the project moves over to a developer. She receives a motley collection of materials from the designer: QuickTime movie files, PDFs, a huge Photoshop file with 51 layer groups… Oh, and something called a “style guide” that helpfully lists the app’s fonts, as well as hex codes for all the colors. (Never mind that iOS doesn’t even understand colors in hex format.)

The developer proceeds to ignore the undecipherable graphics files and moves on to her real work — graphics aren’t relevant until it’s time to put in the final assets. She has been to all the endless meetings about this app, so she knows how it’s supposed to work. It’s all pretty obvious, and secretly she feels those meetings were a huge waste of time.

A month later the first iteration of the app is done. It runs on an iPhone! It has a home screen icon! It receives data over the network! And it doesn’t really look anything like the designer’s expectation. “That’s just a matter of skinning it,” says the developer. “Did you even look at my QuickTime movies?” laments the designer.

What happens next is anyone’s guess. Maybe the project can be salvaged, and it only exceeds its budget threefold while being six months behind schedule. Maybe the designer’s Floating Balls of Delight interface vision gets fully implemented and then gradually scaled down as some voice of sanity prevails. Or maybe the “nearly complete” project just gets quietly dumped when next year’s budget comes in.

The fix for most of these woes lies in prototyping, of course. Prototypes help understand the app’s purpose and its tangible manifestations at all stages of the process. Prototypes can be used to ask questions and provide the team with data to replace feelings and assumptions.

The story I told you above is a caricature that hopefully doesn’t match reality in most app development teams (not anymore, anyway). The reason I described the pathological case is to help explain why it’s very hard for a single prototyping tool to answer all the needs that arise during a development process.

The mobile UI prototyping tools we currently have can be roughly organized into two categories:

1. Clickthrough builders like Marvel, Invision and Flinto.
2. Animation systems like Origami, Form and Principle.

(Sidenote: Framer.js represents a third approach that takes inspiration from the venerable Processing — an easy, visually oriented programming language. I’m counting these out because I feel it’s fundamentally wrongheaded to expect that designers should learn to code in order to contribute to user experience design. I love programming, but honestly I feel that learning something like JavaScript or Swift is merely a huge waste of time for most people. It’s like having to study motor engineering when you just need a car to drive.)

The two categories are highly complementary. Clickthrough builders can be very useful in the initial stages of design. Animation systems are great for working out the polish and detail that can make the difference between a uniquely branded app and a merely functional one.

However, with all these great tools we have at our disposal, it’s easy to ignore that there are still gaping holes left in what can be prototyped. A clickthrough sort of looks like the real thing, but it doesn’t actually interact with the device in any meaningful way. It can’t display real data. It can’t collect data from the user. It doesn’t send notifications or open links in other apps. As a “picture show”, a clickthrough lacks the essential look’n’feel provided by the operating system.

On the other end of the spectrum, new animation systems like Principle are so lovely that it’s easy to get distracted and forget that Your UI Isn’t a Disney Movie (thanks to Sophie Paxton for the great article).

The fundamental structure of an app can’t be represented as a collection of key states, nor a timeline, nor an Origami-style dataflow that represents a rendering loop. These concepts are all derived from 2D/3D animation packages, and while they are useful constructs for representing sections within an app, they are disconnected from the real building blocks of native apps — components provided by the operating system, things like UINavigationController, which feels rather mundane but is absolutely essential for most apps (except perhaps those with Facebook-like unlimited budgets).

Additionally, these tools ignore an elephant in the room: the developer. She’s the one who used to get a dump of QuickTime movies and cryptic Photoshop files… Now she’s getting a web-based clickthrough and a motion prototype that runs on a phone, which is an improvement, but she still has to start from scratch for her work.

There’s room for a third kind of prototyping tool: a design-to-code system. It’s a tool that provides a “sanity check” from the developer’s direction by ensuring that designs can actually be implemented in code. It understands modern operating systems’ UI frameworks and guides designs to leverage their capabilities rather than fighting them with “custom-everything” approaches.

And, because this tool understands code, it generates it too. Complete IDE projects, in fact. Your prototype is now an actual app too, and suddenly it has access to all sorts of interesting possibilities that would be hard to prototype otherwise. For example: what really happens when the user takes a photo and it’s added to a list? How about sharing that image to another app? (A typical UI prototype uses beautiful stock images to stand in for a user’s own content. This is a common example of a situation where a too-beautiful prototype is misleading.)

As it happens, I’ve been working on such a tool with the team at Neonto:

Neonto Studio reached version 1.0 yesterday; even so, it would be hard to call it complete. Like most 1.0 products, it’s more like a useful starting point for something that could eventually become really interesting.

Yet it already does a lot of stuff, some of which is not obvious at first sight. Here’s a couple of highlights:

• Adaptive layouts are an integral part of the design workflow. With instant previews for different devices, you can’t avoid them.
• Data can be mocked up using data sheets and linked to lists and grids. This allows you to design with data (thanks to Josh Puckett for the expression!)
• You can link a data sheet to a web service plugin, and instantly the mock-up data is replaced with real data from the service in Neonto Studio… And the same thing happens in the exported app too.
• Make a button inside a list/grid item, link it to another screen, and you get a detail view automatically. The other screen’s elements are dynamically filled out with the contents of the list row.
• Local data sheets allow you to make lists/grids that are populated by user-created content.
• Assets are generated automatically for both iOS and Android as part of the exported projects. Bring in your images at a good enough resolution, and they’ll get exported as optimized PNG or JPEG (depending on the image data), with various Android resolutions handled too.
• The same goes for fonts, strings and colors.
• Whole folders of HTML+JS+CSS content can be embedded within an app using the web content element + asset folders.

What do you think? Is the “third way” of a design-to-code prototyping tool a worthwhile goal to pursue?