A Scale for Evaluating Mobile Responsiveness
Exploring a new way to measure a product’s progress toward “responsiveness”
The terms “mobile friendly” and “responsive” are far from being a binary measure of whether “you have it” or “you don’t.” Defining mobile friendly on a more granular scale is crucial if we want to improve our mobile experiences. If you can’t measure it, you can’t improve it. This blog post explores a new way to measure mobile web maturity and assesses where Guild Education’s product stands in terms of our own progress.
At Guild, we strive to serve an ever-growing diversity of students. We recognize that our users have varying levels of computer/phone savviness as well a wide range of potential devices (laptops, work computers, slower smartphones, larger smartphones, etc). One of the ways we navigate this varied landscape is by ensuring that our student-facing web apps are responsive. That they’re mobile friendly. That we design with a mobile-first approach. These buzzwords encompass a lot of things:
- A web app that looks good regardless of device size
- Starting designs small and then growing to wider screens
- Choosing breakpoint values to serve the content, not device dimensions
- Faster load times
- Call-to-action buttons that are easy to find and click
These guides and rules are important, and it’s common knowledge that tolerance is dwindling amongst mobile users for slow and clunky sites. Everyone knows that mobile is essential, but not everyone knows what that means exactly. I felt that the concept lacked a skeletal foundation on which to place things such as font sizes, breakpoints, accessibility, mobile performance, and more.
I settled on representing mobile design (and its corresponding technical implementation) as a spectrum of solutions, ranging from completely ignoring mobile to making it the number one focus. Each phase has distinct characteristics that separate it from the previous one.
Every piece of literature around mobile design — and likewise, every app that strives to implement all the buzzwords of mobile design — falls somewhere on this spectrum.
We’ll dive into the phases of this spectrum, assess where Guild lies on it, and address actionable next steps to move our product towards mobile enlightenment.
The spectrum itself:
- Ignore: You built your app pretending mobile users don’t exist. What it looks like on mobile is anyone’s guess. 1920 x 1080 or bust.
- Acknowledge: You anticipated someone might use your app on mobile. You add a “Coming Soon” page or redirect when you detect mobile. Done.
- Handle: You tested your app on a few mobile devices. It’s not a complete disaster. Good enough for now, “it’s usable.”
- Consider: You made intentional design decisions to factor in mobile: layout breakpoints, adjusted font sizes, alternative navigation methods.
- Optimize: You made mobile a priority: you made call-to-actions easier to find, leveraged native inputs, and minimized user journey interactions.
- Supercharge: You made mobile the priority: you reduced bundle sizes, scrutinized load times, tested extensively, and provided native solutions.
Where a product lands on this spectrum is determined as much by technical factors (skills of developers, technology choice, etc) as it is by product and business factors (available bandwidth, company priority, user base, etc). There isn’t necessarily a bad solution — although some awareness of your mobile status is always preferred to none — and the magical end goal will likely be different from company to company. What’s important is recognizing where you stand and being able to identify the actionable next steps to move forward.
Ignore
While its position at the beginning of the spectrum might suggest that a product that ignores its mobile solution is doing something bad, there are some real use cases for a company not to waste mental energy thinking through mobile solutions.
At my previous company, we built software for security analysts to monitor and react to network events. Our apps were saturated with overlapping configuration panels, complex charts and graphs, notification alerts, and tables with thousands of rows. Thinking that our solution could exist on mobile would be foolish. It wasn’t part of our product offering, and no customer would ever seek it. Ignoring mobile was acceptable.
The luxury of ignoring mobile exists for products that have a very specific user segment and don’t risk losing customers by not offering a mobile solution. So you either know that your product isn’t intended for mobile (i.e. acute customer segment awareness) or you have some idea of where you stand on smaller devices.
Mint is an example of a product that has a fully-fledged native app for iOS and Android, but has decided to ignore their mobile web app completely.
Mint allows a user to log in on mobile and just renders their full desktop app. They ignore their mobile solution; it’s less than ideal and not very usable. They invested their resources into a native app and chose not to do anything about users trying to log in to their web app on their phones. Catastrophic? Certainly not, but we’ll see a different way of handling this same use case in the next phase.
Acknowledge
Acknowledging that having mobile users on your app is a possibility is enough to motivate at least some kind of minimal “solution” that could take various forms.
One is to gracefully handle mobile sizes with a landing page that simply informs users that you don’t support mobile:
Another option, often called adaptive design, is to redirect the user to a separate, mobile version of your site if it is detected that they are on a mobile device (m.mysite.com vs mysite.com). Mozilla’s Developer docs recommended against this, calling out the error-prone nature of detecting user devices server-side, as well as the double maintenance of having two different code bases.
Yet another option — if the product has a native app and doesn’t want to build a mobile-friendly browser version — is to have a popup telling the user about the app.
Unlike what we saw earlier with Mint (which also has a native app), Robinhood actually prevents a user from signing in at all on mobile and only renders an informational landing page with a popup about their app. They acknowledge that users might hit their site before their app, and use their site to inform and then direct users towards their app.
Handle
At this point, your team has made an intentional decision to put enough effort into the mobile solution to make it more tailored and usable. It’s not pretty, resources aren’t heavily allocated, and testing is barebones, but the app is usable and adjusted for mobile. Perhaps you ship a heavily slimmed-down feature set, or you simplify your use cases to “desktop” and “mobile” and don’t provide the granularity of tablet and older smartphones. You haven’t invested enough resources to establish reusable patterns or redesign every feature, but you have ensured the critical portions of the app fit and that the experience isn’t one of obvious neglect.
Consider
At this phase, you have placed a much heavier emphasis on mobile use cases. Key things are implemented like increasing font sizes, removing horizontal scroll, and restructuring navigation. Every feature also has some minimal specs for mobile implementation. Horizontal dropdown navigation isn’t just resized to fit across a mobile screen; it’s replaced with side navigation that stays tucked away. Inputs are made bigger to attract fingers more easily, and paddings are adjusted to make content more breathable.
Developers are likely to be using the mobile inspect mode in browsers allowing them to test their UIs on several screen sizes quickly. Developers are also laying groundwork for common patterns and reusable UI infrastructure to establish mobile consistency. Breakpoints are consistent across the app, fonts readjust in unison, and behaviors like buttons becoming full-width and sticking to either the top or bottom of the screen are commonplace. Considering the mobile solution is just as much about design decisions as it is technical investment in reusability and consistency.
Optimize
At this phase, you’re no longer just making sure the content fits, but that you have the right content.
You can always make a table fit and be navigable by scrolling, but is a table the best way to present data on mobile? Yes, a user can scroll to the top to always find the submit button, but will the user want to scroll every time? You can easily save on space by making extra content collapsible, but is that extra content even necessary?
No longer just the layout, the user journey is now also redesigned through the lens of someone with:
- two thumbs (and many of us even prefer using our phone with one hand)
- drastically shorter attention span (we’re often distracted when on our phones)
- potentially less time to achieve a task (we rarely spend several minutes trying to do one thing on our phones, although exceptions to this exist)
- a trained expectation of performance (other fast mobile apps like Google Maps and Instagram set this bar high)
With this new persona in mind, a lot has to change. Redesigning existing content to be presented differently, removing/consolidating steps in the user flow, streamlining any longer tasks with animations and transitions to keep ever-fleeting user attention, and putting call-to-action buttons front and center.
Things like information density are taken into consideration. A user on their phone may not expect encyclopedia-level information; they may want the bare minimum needed to accomplish their task and move on. Other users who don’t have access to a computer, and a phone is their only way to view information, might actually want that detailed wall of information.
Optimizing for mobile means reconsidering the breadth of users you have (first time users, power users, desktop-less users, etc.) and designing for each within the realm of mobile. The user personas that are kept in mind during architecture, design, and product meetings now must also include permutations for mobile devices. Every pixel (copy, buttons, animations) is scrutinized and intentionally placed given the limited screen real estate, and entire experiences are designed specifically for the mobile user.
Diagrams such as the one below are kept in mind to optimize for finger placement connivence. This may seem overkill, but if you are optimizing for the mobile user, every detail adds up.
Supercharge
At this point, mobile is the focus, and it goes way beyond just building responsive design layouts that fit nicely. Performance, load times, and network latency are all now crucial. A site cannot achieve mobile superiority if that super slick, fluid, well-animated app takes 9 seconds to load on a user’s device.
At this phase, a substantial amount of time and effort is spent on speed. Things like bundle size, the number of requests a UI has to make to the server (each one incurring network latency penalties which can add up), and overall CSS paint time and JS processing time become important metrics. Testing occurs on a plethora of devices. Lighthouse (a Chrome performance audit tool) is used to simulate slower processing power and 3G speeds. Caching strategies and bundle optimizations like code splitting and tree shaking are must-haves. Content that gets downsized visually (images, banners, logos, icons) to fit on mobile should also be downsized physically (in terms of bytes) to help reduce load times.
It wouldn’t be a stretch to say that keys to achieving this performance level include CI checks that ensure bundle sizes and page load times remain below certain thresholds. A code change that drastically increases loading time is blocked for further review. As mentioned earlier, mobile is the focus, and everything from the release process to server architecture to content delivery networks is optimized with mobile speed in mind.
Responsive design isn’t complete just by resizing/reshaping your page content and optimizing for thumb access. Responsive design should be seen as encompassing design. How wide of an audience can your product serve? With performance bottlenecks, this audience shrinks. Page content doesn’t matter if the user never sticks around to see it.
So now comes the big question: where are we at Guild?
Where Are We at Guild
We will dive into Guild’s various mobile aspects as they stand today, but right out of the gate, one of the biggest hurdles we currently have is performance.
Performance is not necessarily measured in bundle size, download times, number of caches, Time To First Paint’s (TTFP), or animation speeds. While these are all important metrics, they don’t necessarily represent the user journey. Our user journey can (and in my opinion should) be seen as the time spent from the moment a user enters the funnel to the moment they have achieved or (nearly achieved) some measurable metric, such as application start rate.
Currently, one particular user journey at Guild can include up to four steps:
- User loads and then signs in from our Partner Page (information about Guild’s educational benefits)
- User is then redirected to our Student Home Page (launching spot for logged-in users)
- User then navigates to the Catalog (our list of academic programs) to search for programs
- User finds a program they’re interested in and clicks Apply (Application page)
Users will spend a minimum of 24 seconds* waiting on those 4 pages to load. Not 24 seconds exploring, thinking, navigating, scrolling through options, becoming informed, learning about Guild. 24 seconds of loading time. That’s over a WiFi connection on a MacBook Pro.
A supercharged mobile experience is impossible if a user spends at least 24 seconds staring at loading pages in a world where average web page visit times are 15 seconds.
Which brings me to Guild’s position on this spectrum: In my opinion, we are currently in the process of optimizing.
We are well past simply considering the mobile use case. We have done research to collect the most often used devices. We test extensively on multiple devices (and even have a device cart at the office containing older smartphones and tablets). Newer features have designs uniquely tailored for mobile experiences, and navigation methods exist for both use cases (desktop and mobile).
Are we perfectly optimized? This fluctuates and differs across the organization, but there is always room to improve. And are we supercharged? Not yet. The 24-second, 4-page journey is simply too convoluted and slow to be considered a supercharged experience, and we haven’t really tackled our bundle sizes in an intentional manner.
Next Steps
There are a lot of ways our organization hopes to move towards the supercharged state. We’ve already identified some lower hanging fruit, such as increasing our Cypress testing and implementing certain webpack strategies, and we know that the growing number of separate React apps have to be eventually be consolidated.
Stay tuned for a much deeper discussion into that exact question — how do we get from optimizing to supercharged — where we will dive into technical details, architectural decisions, and ways to navigate the business/product aspects. Part 2 of this blog post is coming soon.
Until then, we’ve seen how mobile/responsive design is better represented by a spectrum than by a black-or-white evaluation. Every organization is somewhere on this spectrum, and improving your offering and ultimately serving more users requires a strong understanding of the current state of your mobile product.
*5 rounds of tests were done recording events on the network tab and measuring perceived time (subtracting time spent moving mouse to navigate) during the navigation of those 4 pages
