(Company names omitted to protect privacy.)
By: Jackie Tanner, UX Designer
Between Dec. 09, 2015 — Dec. 11, 2015 a smaller company collaborated with a Big Company in an international, 5-person, qualitative, moderated user study to determine the effectiveness of the smaller company’s onboarding flow and tools as it currently stands. There were many purposes to this user study, including:
- The Big Company’s need to obtain insight on their introductory email copy
- Determine whether any UI design flaws exist in initial survey
- Observe a user’s 1st time interaction with a smaller company’s site
- The Big Company’s need to determine whether users can find, complete and understand the purpose of Check-Ins
What We Learned
Overall we found that our tool is moderately easy to use with all 5 users rating it 3+ on a 1–7 scale, 7 being “very easy”). Most users (⅗) used Internet Explorer (as expected when compared to Google Analytics of our current user base), the remaining 2 users were on Firefox and Chrome. Although it wasn’t being tested, we also found that all users were able to logout without any difficulty whatsoever.
- Email: Almost all users (⅘) stated the need for the introductory email to contain “action-item” or “mandatory” in the subject header and a link/button to the smaller company’s site within the email body.
- Survey: Most users (⅗) stated some form of misunderstanding of the difference between the progress bar vs. countdown timer. Some users (⅖) experienced difficulty with the “Next” button falling below the fold, due to their screen resolutions. Almost no users (⅘) have any interest sharing their results on their social media.
- Onboarding: Most users (⅗) ignored the tutorial, were able to find the Help “?” button in the header, and expressed some level of interest in viewing a tutorial at a later time.
- Check-Ins: Almost all users (⅘) seemed to have no confusion about the purpose of Check-Ins overall. Most users (⅗) had some confusion regarding the privacy settings, be it UI design-wise or the terminology (⅗) or outlier (⅕) questions about whether upper management has access and if the conversations will take place somewhere in the platform itself.
We gained a lot of valuable insight from this session of user tests. Though we had some confirmed successes (ex. Help button, logout, understanding of Check-Ins, ease of use, etc). The key areas of opportunity discovered from this test are:
- Urge future clients to include some form of “action-item” copy in the subject headers
- Also recommend strongly they include a clearly labeled link/button to the smaller company’s site in the body
- Separate the progress bar and timer (so it’s clear they’re 2 separate forms of measurement)
- Address the issue of some users losing the “Next” button below-the-fold
- Remove ability for users to ignore the tutorial entirely
- Include the ability to relaunch tutorial whenever users choose
- Include logic that will allow for different onboarding flows for various user personas
- Change privacy settings UI to clearly express on/off states
- Remove ambiguous copy and state purpose(s) more clearly
Over the course of the following few months the development team will build these changes. We will determine our successes and failures with a new series of user tests upon completion. Stay tuned!