Why We Watch Users
This is an ancient blog post originally published by Brian Duff on 6 August 2003 on Radio Weblogs, and rescued from the Wayback Machine. The images from the original post have unfortunately been lost to time…
I‘m not a usability expert, but I’ve been fortunate enough at Oracle to work closely with very talented usability engineers on the areas of JDeveloper I’m responsible for.
From time to time, we invite customers to come to Oracle and participate in usability tests so that we can get quantitative data about whether functionality is easy to use. As a developer, sitting in on these tests (developers are kept safely hidden away from customers on the other side of a one-way mirror) has been hugely beneficial in the past. I’ve often been surprised to find that a feature or dialog I thought would be pretty straightforward stumps three out of five test participants.
Often, the solution to such usability problems become readily apparent when you’re sitting watching a customer try to figure out how to use the tool. Many times during these tests, I’ve had those real “light bulb” moments, discovering the way I should have designed something in the first place.
It’s almost impossible as a developer, or even as a user interface designer to avoid letting your familiarity with the tool or the user interface patterns in your product cloud your impression of how new users will learn to use your software. At least, in the case of JDeveloper, we can make some basic assumptions about the technical experience and tolerance of our users. Developers tend be willing to experiment a little until they figure something out, and are usually not as susceptible to the fear that novice computer users experience: the terror that every mouse click could bring it all crashing down. It must be incredibly difficult for the developers at Microsoft to design a product such as Windows that has to cater to the complete novice user all the way up to the super-power users that developers are (or at least like to think we are).
The last week or so, I’ve been visiting my parents in Edinburgh and providing a bit of basic Windows tuition. My mum is completely unsavvy when it comes to computers. She’s only just getting over that initial fear of bringing it all down with a single click (she still has a bit to go, if the audible horror whenever an accidental click brings a new window to the top is anything to go by). They’re planning to scoot over to France or Spain for a few months later in the year, and now have a laptop, loaded up with Autoroute and Windows XP.
It was initially quite an erm… dictatorial… experience, with me saying things like, “if you want to see the roads near dijon, you can drag a box like this, then click inside it like that to zoom in”, with my mum diligently trying to keep up on the mouse. This wasn’t proving to be a very successful method of tuition… I’m pretty sure I won’t be working for Oracle Education any time soon…
Near the end of the day, my mum asked how to switch the computer off (it’s an old, second hand laptop with no funky power features). I’d already covered this about five times already, with my mum initially expressing the familiar surprise that a button called “Start” leads to a menu that turns the computer off. So I decided to try a different tact. “Why don’t you see if you can figure it out?”
A Windows Explorer window was on screen at the time. Fairly quickly, she spotted the Help menu, which seemed like a good candidate. The help menu has three items in it:
(missing image of Windows help menu showing three items: “Help and Support Center”, “Is this copy of Windows legal?” and “About Windows”)
Rather than going for “Help and Support Center” (the correct option), she went straight to “About Windows”, and was then confused by a dialog box that displays copyright information and tells you how much physical memory Windows has.
I’ve recently been reading About Face 2.0 by Alan Cooper. At this point, I recalled that this book actually specifically cites Help->About as being a problematic menu item to have in the Help menu, because it typically doesn’t actually provide any helpful information about the product. I was skeptical about how much of an issue this actually was until I saw it in action. Thinking about it, it’s quite a natural choice. She wanted to find out about shutting down windows. About Windows sounds like it would be more useful for doing this than the Help and Support Center, which frankly sounds like something you have to pay for.
Having redirected her to the correct place, she was confronted with this:
(missing image of Windows Help and Support Center)
She immediately expressed some horror at the sheer amount of text on this window. Actually, having never really used Windows XP’s help before, so was I. I asked her if there was anything obvious she could use to help her find out how to carry out her task and she said, “I have to read through all this text first”. We both found it distracting that after a second or so, a few paragraphs more text appeared under the “Did you know?” section.
The engineers who designed this knew that most users would end up needing to use the Search control to find the specific thing they were looking for. They made its label really big, put it at the top of the dialog, and made the background behind it a different color. I think it really stands out. But the amount of text in the area below still seems to distract users. Or my mum at least, admittedly she doesn’t constitute a decent test sample :)
Eventually, she spotted the Windows basics bullet, and clicked on it. Shutting down the computer is pretty basic.
(missing image of Windows Help and Support Center after clicking Windows Basics bullet)
My first impression on seeing this was that my mum would never figure out that “Core Windows tasks” was probably the place to go. The words “core” and “task” seem pretty technical for a section of the help dedicated to completely new users. She’d also probably miss the subtle cross in the icon indicating that this is in fact a folder containing subtopics. Indeed this was the case, and she kind of stared blankly at this screen for a few seconds before I suggested clicking on the “Core Windows tasks” item.
(missing image of Windows Help and Support Center showing Core Windows tasks expanded in the Windows basics section)
Once again, this provoked a blank stare. I later found out that had she clicked on “Logging on and off Windows”, the fun would be over, and we could go out for dinner. Having scarcely used Windows herself alone, never mind in a multi user environment, logging on and off sounded to her like something vaguely related to trees.
At this point, I finally relented and suggested using the Search control at the top of the window. Painstakingly, she typed out “closing down” in the search box, and very quickly figured out that the big green button with an arrow on it was the thing to press.
(missing image of a big green Windows XP Go button)
I’m a new convert of these big green “Go” buttons in XP. Now I see why Microsoft chose to use them instead of plain old text buttons. It’s obvious that you need to click on them to Go. You don’t have to read the text to make sure it’s doing to do what you expect it to do.
I fully expected it to all go horribly wrong because she’d typed “closing down” instead of “shutting down”, but now something really impressive happened:
(missing image of Windows Help and Support Center results for “closing down”)
The search capabilities in Windows help system actually seem to be very good. This was spot on, and better still, it was refined. We didn’t get all 300 topics in the whole of the help system containing the words “closing” and “down”. We got exactly what we were looking for.
However…
(missing image of Windows Help and Support Center topic for “Turn of the computer”)
This initially looks great. There are specific instructions about how to shut down the computer. The problem is, the instructions are wrong. It looks like someone forgot to update them when they changed the look and feel of Windows between 2000 and XP. If you display the Start menu in Windows XP, there is no Shut Down. It has been replaced with Turn Off Computer (much better, shame the help wasn’t updated). Clicking it also doesn’t display a drop-down list.
(missing image of Windows XP start menu showing the new “Turn of Computer” item)
I hope this article doesn’t read like a jibe at Windows or Microsoft. Here’s the thing about this process… at almost every stage, I felt that Windows had been designed in the way I’d probably design it to be intuitive for new users. In many ways, it works far better than anything I would have designed. It’s also a bit of an unlikely situation. My mum would eventually have found the Turn Off Computer item by accident in the Start menu. Perhaps if the help topic had been correct at the end, I would have been a bit more impressed. After all, we got there in the end.
There isn’t really a moral to this story. I just found it interesting to watch, and felt it was worth sharing. It reinforced my belief that watching users figuring out how to use a product is incredibly valuable, and reminded me that software companies (Oracle included) should always carefully check help topics and make sure they’re up to date.
Modern notes from Future Brian:
I was 26 years old, 5 years into my first job on the JDeveloper team at Oracle in Reading, United Kingdom. I clearly remember writing this post with a can of Kronenbourg lying on the living room floor in my parents house late at night in Edinburgh on my Dell Inspiron laptop. I had been in Edinburgh for the summer hanging out with my family, and was just about bored enough to try this blogging thing (which was becoming immensely popular) for the first time.
I was 100% a Windows user back then, something that’d change significantly in the years to come. Windows XP still seemed relatively new, having been released in 2001. Windows XP was quite widely panned for its cartoon-ish brightly colored UI, which was a massive departure from the bevelled gray rectangular look of its recent predecessors. For my part, I actually liked the new UI a lot.
My parents weren’t remotely computer literate; at the time I wrote this my mum and step dad had just recently retired from about a decade running a take away food shop in Leith, avoiding all attempts by me throughout that time to modernize their record keeping using these shiny computer things. The immense rise of the web over the previous few years changed everything, and my mum had her first Windows laptop so she could keep in touch with folks while she traveled. She was actually terrified by the thing, so I spent some time in that summer helping her to acclimatize to it.
This blog post probably suffers somewhat from having lost its images, but it did demonstrate a pretty bad UX issue for newbie users like my mum in Windows XP. A few years before this, in 2000, I’d taken my first trip to Oracle USA to participate in a one-way-mirror UX session on JDeveloper, and so I was quite interested in improving user experience by observing behavior, and I think that’s what led to this blog post.
As with most of the things in these archives, it’s hopelessly outdated, but through this and other experiences, I learned a lot about the fact that I’m not the target user, and how hard it is to design software well for every type of user.
