Guerilla Usability Testing of Hootsuite Onboarding

On May 27, Samuel Hulick gave a talk at Hootsuite’s Vancouver office, explaining the importance of smooth and compelling onboarding experience for new users of a product. This inspired me to run a quick usability evaluation of the onboarding experience on the Hootsuite web app: hootsuite.com/dashboard.

Disclaimer: I don’t work for or represent Hootsuite. I’m just passionate about Interaction Design, and always looking for ways to improve the user experience :)

If you are not familiar with Hootsuite’s web interface, here is how it looks during onboarding (click to expand):

The main interface is covered by a dark half-transparent overlay. The current element of focus is highlighted in white, while a pop-up guides the user through a series of tasks.

Participants

The study was conducted in the department of Computer Science of the University of British Columbia. 6 participants were graduate students, in their mid-twenties, and one was a faculty member. All participants except the latter were actively using multiple social networks: most commonly Facebook, Twitter, and LinkedIn.

Note: These participants’ demographics only reflect part of the target audience of Hootsuite: casual users. It does not cover key stakeholders like social media professionals and businesses, who are more likely to pay for a premium plan. However, for the purpose of a simple usability test, casual users of a product are enough to uncover a variety of usability issues.

Tasks

After a brief introduction on Hootsuite, participants were asked to sign up for Hootsuite, using a web browser on their laptop (Task 0). Participants then followed the onboarding process, which comprises of the following four tasks:

1. Add social networks
2. Add streams
3. Create a new tab
4. Schedule a post

Since these four tasks are put forward by Hootsuite’s designers, we can reasonably consider that they are the core value propositions of Hootsuite: aggregating social media and posting content. It therefore makes sense to focus our user study on these four critical tasks.

Note: Depending on the behaviors observed, some participants were asked to complete extra tasks at the end, to evaluate their understanding of the interface. For instance, participants were asked to add a stream from a different social media if they had been using the same social media throughout step 2.

Procedure

Participants were recruited with chocolate. They used their own laptop whenever possible, while I was observing them and taking notes. I asked participants to voice their thoughts as they were were interacting with the software, following a classic think-aloud protocol. To preserve anonymity, participants’ names were replaced by P1, P2, etc. I then performed a thematic analysis, identifying common issues and highlighting them in each participant’s transcript.

Findings

Some important usability issues were found for each of the 5 tasks introduced above. These issues are presented below, in the same order that participants encountered them.

Task 0: sign up for Hootsuite

“Oh my god this is hard! Do I have to pay for it?” (P3)

Issue: 4/7 participants had trouble signing up for a free account.

The current signup process emphasizes comparing the different plans available, whereas participants were looking for a simple login/password signup form. In particular:

• On hootsuite.com, clicking on Sign Up actually opens the Plans page.
  This might cause a negative transfer effect: on many websites, pricing information is different from the signup form.

• On signup.hootsuite.com/pro-ent-na-english-r9 (the link promoted by an add on Google), there is no sign up button, but only “Compare plans and pricing”. P3 tried to click on Get Started Today!, which is not clickable.
  P3 eventually googled again “hootsuite” to find another sign up page.

Recommendation: This issue might come from our participants’ demographics (casual users), whereas Hootsuite’s sign up process might be optimized for paying customers. It would therefore be unwise to make definite recommendations without more data. A general approach would be to focus on helping users create a simple account as quickly as possible, and then present the different plans and let users select one to finish the signup process.

Task 1: add social networks

“For some reason it only has Facebook, I wonder what happened to LinkedIn…” (P1)

Issue: 5/7 participants didn’t realize that connecting to a social network was different from importing one (or several) account. As a result, these participants failed to import their account from at least one of the social networks, but were convinced that they did. Later they were confused or frustrated because they could not find the social network they thought they added earlier.

The root cause of this issue is that participants seemed to equate a social network with their own account on that network, without considering the possibility of pages, groups, etc. It is worth noting that the situation might be different for social media professionals, who are used to managing multiple accounts and pages.

Recommendation 1: When authorizing a third-party app to access their social networks, participants seem to click through the different modals very quickly. When they finally reach the “Import” modal of Hoostuite itself (see screenshot), most participants simply clicked the “Finished Importing” button, without realizing they had to manually select which accounts/pages/groups to import. Two possible solutions:

• In the “Import” modal, have the first account selected by default.
• Keep the “Finished Importing” button greyed out until users select at least one account/page/group.

Recommendation 2: The “Add Social Network” wizard found in the main application should indicate which social networks are already connected / authorized. So far, the wizard only shows the option to “Connect With X”, even if X is already connected. Instead, this tab should show which accounts/pages/groups are currently imported from X. It could also be helpful to group the social networks tabs in the left sidebar in two categories: those already connected, and those not yet connected.

Task 2: Add streams

“I have no idea what any of these buttons mean” (P1)

Issue: Although all participants managed to add 3 streams (as instructed by the onboarding tutorial), at least 4/7 didn’t fully understand what was happening. In particular, multiple participants did not notice that they could select a different social network via the dropdown in the “Add a stream” placeholder.

Left: Current placeholder for adding a stream. The social network currently selected is not very visible. Right: Suggestion: Users explicitly pick a social network before selecting what to stream from it.

Recommendation: The social network dropdown could be made more salient, emphasizing the social network currently selected. Indeed, the profile picture displayed might not be sufficient to identify the account, especially for companies that use the same logo everywhere. A few possible solutions:

• Color the entire dropdown rectangle with the main color of each social network.
• Use a 2-step process in the “Add a stream” placeholder:
  - users first choose a social media account (cf. concept picture on the right),
  - then what they want to stream from it.
• Modify the onboarding tutorial to explicitly require users to select a different social network (if they have connected more than one to their account).

Task 3: create a new tab

“I thought I added three streams. Why are they not here?” (P3)

Issue: Similarly to Task 2, participants were able to complete this task (as instructed by the onboarding tutorial), but at least 3/7 thought they were renaming the current tab instead of creating a new one. As a result, they were very confused at the end of the tutorial to see that all the streams they added had “disappeared”. In fact, the tutorial instructed them to add the streams in the first tab (the default one); but at the end of the tutorial, the second tab is shown, and it is empty.

This participant did not realize that they were creating a new tab, and named it “my first tab”.

Recommendation: The simplest solution might be to simply remove this step from the onboarding tutorial. Unless I’m mistaken, the ability to organize streams into different tabs is not key for being able to use Hootsuite (contrary to adding social networks, adding streams, and cross-posting). In addition, tabs are a fairly common mechanism, so we can expect users to figure out how to use them on their own. A few participants mentioned that they would prefer a more salient visual design for the tabs, instead of the current rectangular shapes and shades of gray that are found throughout the interface. P6 adds: “there’s just so much stuff crammed up here that it’s hard to see which tab you have selected”.

Task 4: schedule a post

“It really wants me to install this extension, are you sure I don’t need it?” (P5)

Issue: 2/7 participants had trouble scheduling a post.

P5 added a “Scheduled” stream, and tried to use it to schedule messages. They were confused by the “Install Extension” recommendation in this stream, and thought it was the only way to “share and schedule links”.

Although P4 expanded the “Compose a message” dialog at the top of the window, they failed to realize that the calendar icon would allow them to schedule a post. Instead, P4 thought the calendar was “just another Twitter option”, like adding a file or editing privacy settings.

Recommendation: These issues were experienced by too few participants to assess their severity accurately. There are a couple of small changes that can be explored, such as:

• adding a “tip” card in the default “Scheduled” tab to explain how to schedule a post from within the app;
• making the calendar icon stand out visually from the other icons;
• replacing the calendar icon by a timer icon, to be consistent with the icon used on the “Scheduled” stream (or vice-versa).

To reduce the number of visual elements present at any given time in the interface, the current “Compose a message” toolbar could be replaced by a single “Compose a message button”, which would then open a dialog.

Some more thoughts on onboarding

In his presentation, Samuel Hulick argued against the use of a half-transparent layer with instructions overlaid on top of the application interface. In his opinion, it separates users from what they really want to interact with: the application itself. My participants seem to agree:

[The tutorial] has blocked out so much of the context that I couldn’t do it again. It’s gonna get the job done, but it’s not teaching me. (P1)

I would have just clicked through all of this (the tutorial) and figured all of it by myself. It’s like apps: when they release a new version, you have five screens to swipe left through. I want a button to say: I don’t care which features you’ve added, I just want to use the app! (P2)

I don’t read those [the popups]. I just discard them. (P3)

I’m kinda assuming that it’s going to be easy enough to use without reading the popups. (P6)

There are many possibilities instead of the layer + instruction popups approach. For instance, the instructions could appear in a “Hootsuite tutorial” stream displayed by default in the first tab. New instructions would appear in the stream as users complete various actions in the interface. The benefit of such an approach is that users are in control of their own learning: they can play with the interface without any instructions at first, and can read the instructions stream to learn more later on.