Let the User Testing Begin!

Posted on behalf of sarika joglekar and Ishita Ferdousi.

Last week, we wrote about creating the journey maps and personas of users based on our findings. We described the specific issues that users might be facing. Now, we are working on improving the usability of the most common use cases.

In this process of redesign, we have started wireframing and proposing different ways to tackle some of the problems discovered. Wireframing is an important step in any screen-based design process. It will allows us to define an information hierarchy and roadmap of functionality.

In order to get a clearer picture of the different ideas we were considering, we divided our class into two teams and approached the problem in different ways. This gave us the opportunity to test alternative design decisions.

The first team — sarika joglekar and Pratik Joglekar — designed a classic approach and tested the following

• The navigation at top with drop-down to reveal version history
• Open table of contents with visual mark to locate section
• Use of icons to indicate link function within a document
• Need of global search for searching specifications
• Use of category tags w/ specification name

The initial wireframe created by Pratik Joglekar and sarika joglekar for the W3C specification project.

You can check out the live wireframe here.

We did a first round of usability testing with two users. We asked about their first impression of the visual style, layout and iconography. Both participants found the design comprehensible and clean and the layout, colors, contrast effective. We then gave them certain scenarios and asked them to think aloud about their experience.

We were interested in insights on certain key actions like accessing Github to file a bug, switching between different versions of the specifications, and searching for other related specifications. Our testing gave some great insights about what the user needs. Some key findings from this testing revealed as listed :

• Global search: Global search will work when the user knows the parameters. The header is taking up a lot of space leaving less room for the documentation itself.
• Links (Icons) within spec: The links accompanied with icons was breaking the flow of reading and hence was increasing cognitive load.
• Table of contents highlights to locate section: Our participants recommended we take a look at specification that has an absurdly long table of contents, just to make sure the idea works for all specs.
• Icons on the top (Header): The icons at the top were well received and they specifically liked the ‘File a bug’ icon but found ‘More info’ a little confusing, since it was not a call to action.
• Changing version from drop down: It was a little difficult to figure out that the drop down offers different versions, hence it would help to write what action to expect right at the top.

We iterated on the recommendations and came up with an improved set of wireframes and sent it out to our testers again. One of the concerns pointed out was of responsive design and how the on hover links will work on mobile. Most of the other improvements we did were well received.

“Love the dropdown for version. Also like the collapsable side menu. My only concern is with the hover text. I often browse the spec on my phone — so I don’t have a mouse pointer. How does it work in that example?”

The second team — Ishita Ferdousi and Raeesha Altaf — designed with a modern approach and tested the following:

• Use tabs for nav, putting the boilerplate and history on tabs, also testing the toolkit on right rail
• Use an accordian for the table of contents
• Collapsible table of contents

A screengrab of Ishita Ferdousi and Raeesha Altaf’s wireframes for the W3C Specification Project.

We conducted user testing with three users who are working with the W3C. We showed them our wireframes and asked what they thought on the overall layout, navigation between different versions of the same specifications, boilerplate content, table of content and tool buttons. Some of the insights that we got are as follows:

• Overall layout: First impression is really good. Things are more structured than before and it was time to get those icons on the specs. But check for long titles and work more on the real estate of the page.
• Version: We rightfully managed to address the problem regarding the versions. But think about including all the other available versions and give the users cue to let them know that other versions are available.
• Boilerplate content: Users like the tab idea. They understood that tabs are organized and put together according to the nature of the meta data.
• Table of contents: Content at a glance was appreciated by the users. The idea of scrolling and the visual cue following you at the table content would be very nice. But the triangle doesn’t feel like a cue to collapse and open the table of content. Also, how can we see the whole content whenever we want?
• Tool Button (icons on the right): Users love the icons. Users believed that it was high time that those icons were put in the specs for a more visual experience.

“The buttons are fantastic. We can argue what buttons we need or not. But, I love the idea.”

We are in the process of synthesizing all our findings and creating a combined design from the information gathered during user testing. The next step will be restructuring and refining the wireframe and moving towards visual design for web, mobile and print.

Thanks for following along. Stay tuned!