Why designing icons is not so trivial
As a Graduate Designer, creating and running user tests from scratch is something I had only done a few times before. When I did they were more likely to be on friends, rather than real end-users. Recently, I was involved in a project to update the icons for one of our products. This would be the perfect opportunity to experience how valuable research can be for validating designs.
The problem — what do those icons mean?
Researchers as far back as Jakob Nielsen have proven beyond any reasonable doubt that recognition is easier than recall. In a nutshell, it’s a lot easier for users to see something and recognise it as something familiar, than asking them to remember something without a cue.
Let’s start with an example. What do you recognise the below icon as?
If you said “upload” or “share”, then you would have a problem in one of our products, Stash. If you said “pull request”, then congratulations and thanks for using Stash — but you probably identified this because you have seen it before and have recalled it. For a new user it does not really relate to the core concept of a pull request, which is about reviewing and merging code. Instead, many people seem to interpret it as an upload button.
An icon’s job is to convey the meaning of an item or action reliably for the intended audience. Sometimes, this can be critical — if designing for the user interface of medical equipment you wouldn’t want the ‘off’ switch to look like the ‘administer life saving electric shock’ button.
We shouldn’t try to be revolutionary in our design of icons just because we can. At the moment, some of our icons differ significantly from those of other Git services— and as many of our users have had previous exposure to their icons, they may be challenged in Bitbucket. This is what you call ‘external consistency’.
The way I tested the iconography
Step 1: Triaging, brainstorming and sketching
My first step was auditing all the icons we were using, and categorising them into icons we should redesign, or ones that simply needed to be recreated in the new 1px style which came about, partly to match our illustration guide — and for line weight consistency. I then went about sketching some ideas for the icons that needed to change.
Step 2: Going high-fidelity
Once we felt confident with those concepts, I converted them into pixel-perfect 20x20 px versions. It’s important to keep in mind the context of an icon, it could look great on its own, but as part of a group of icons, it might stick out like a sore thumb.
Sometimes, jumping into high-fidelity mode too early can be a bad move. You can get stuck pixel-pushing a solution that wasn’t effective to begin with. On the other hand, staying in whiteboard sketching mode for too long also has drawbacks: lots of changes needed to be made when I tried to convert big hand drawn sketches into 20x20 pixel graphics. There was simply not enough real estate for drawing complex shapes.
Step 3: Testing usability
I then arrived at a set of icons that I thought were improvements over the current solutions. But gut feel should not be enough to base a decision on for something that would affect literally every user of Stash. Shipping a misleading icon would lead to our users getting confused. This would break one of our core values at Atlassian, Don’t #@!% the customer. With guidance from some of the more experienced UX Designers, I began writing up a plan for testing the icons amongst developers who were familiar with Stash or Bitbucket.
The test consisted of three parts to test the icon’s identifiability and hence its effectiveness:
- The icons were presented without context, purely isolated on a sheet of A4 paper. The user was asked “Within the context of Bitbucket or Stash, what do you think this could mean?”. I then showed each icon to the users separately, repeating the question every time.
- The icons were shown as part of the sidebar design. This gave them the context of where the icons would live. The users were now asked to identify which icon they would click, given a scenario, such as “Imagine you want to view the files of a repository — what would you click?”
- To gather some more qualitative feedback, I asked participants about what didn’t work for them in cases where there was a mismatch between the intended meaning of an icon, and the user’s interpretation.
Step 4: Analyse the results
I summarised the results into a table which when viewed as a collection allowed me to quickly see which ones of my icons were recognised correctly, and which ones weren’t.
The more red cells you see within a row, the worse an icon performed.
Outcomes
What worked?
Running the user sessions gave us insights into user thinking that we would have otherwise missed out on. These quick 20 minute sessions allowed us to either validate or invalidate different designs quickly so that they could be iterated and improved on.
Running the sessions 1-on-1 allowed them to go quickly and focus on the important details, and it allowed me to see first-hand how participants reacted when trying to identify the icons. This sparked some interesting discussions later.
In the second part of my test, all participants were able to perform tasks like “Create a pull request”, “View the files in this repository” and more, without needing the complementary text of a label. This is good to know that these icons still worked in the likely real world scenario of a collapsed sidebar.
What didn’t work so well?
It turns out icons tend to be pretty hit-and-miss when viewed in isolation, without any application context around them. People either get them, or they don’t.
For example:
0/6 participants recognised the current Stash ‘repository’ icon correctly, despite 3 of them being Stash developers. Most thought it was to do with a single commit or branch.
This was my proposed new ‘repository list’ icon. This was interesting as once again 0/6 recognised it as a list of repositories. However, all participants found that the 3 tiered representation was communicating the idea of a list. When asked for suggestions on what could represent a repository better than the git logo, every user gave a different response, ranging from code symbols, to a cylinder. Coming up with a well accepted icon for a repository has probably been the hardest one of the lot so far.
In retrospect, I should have included a lot more of the current icons so I could see if my proposed solutions work better than them. However, people who have prior experience with Stash would skew the results as they are already familiar with them.
What’s next?
I am working to get the icon set internally tested via dogfooding. After collecting feedback and iterating, we can hopefully deploy the new icons to our customers some time in the future.
Did you enjoy this post? Want more of the same? Consider following Designing Atlassian.
