Don’t just talk, set an example
Rolf Molich on usability, communication, and certification
“You may have never heard of Rolf Molich. Yet, if you’ve done any usability testing, design evaluations, or heuristic inspections, then you’ve been affected by his pioneering work.”
This was how Rolf Molich was introduced in a blog post on Jared Spool’s UIE website back in — wait for it — 2003.
Based out of Denmark, Molich has such accomplishments from pioneering the heuristic evaluation technique with fellow usability luminary Jakob Nielsen, to helping create the international UX accreditation course . He was awarded the 2014 UXPA Lifetime Achievement Award for his work in the industry.
I spoke to him about his work, the changes he’s seen in the industry, and the need for UXers to create a common understanding of the value of their work.
You’re based out of Denmark, which means that some people may not be aware of your work. What are some of the key things you’ve worked on?
The accomplishment that I am most proud of is teaching introductory level UX courses at the University of Copenhagen so well that my 250 students in 2013 named me “teacher of the year” for my highly useful approach to teaching.
I’m also proud of The Comparative Usability Evaluation (CUE) studies, which over 15 years provided data about the effectiveness and efficiency of usability testing and expert reviews carried out by industry professionals. The results have been published in several refereed articles. The most important result is that in most cases the outcome of a usability test depends heavily on the moderator.
I have also helped to create an international UX certification program CPUX, in particular the foundation level and the usability testing curricula. I was co-editor on both curricula and they’re publicly available.
You’ve been a usability practitioner since 1984. A lot has changed in this time for tech and what was then called usability engineering. What are your thoughts on what’s changed since then?
I can’t think of any revolutionary surprises, but I am pleased to see a lot of healthy evolution and increased understanding of usability and user experience concepts and methods. Most of the methods we use today were already around in the 1980’s, but of course today we have a much better understanding of these methods and their potential pitfalls.
I have of course been surprised to see the technological advances that allow for example eye-tracking, remote usability testing and unmoderated usability testing at low cost.
You were one of the original creators of the heuristic evaluation, though this doesn’t seem to be used much these days…
The 10 heuristics that Jakob Nielsen and I conceived in the late 1980s are still highly useful for preventing usability problems. My dream was that the heuristics would be used to brainwash designers so we would never again see user interfaces that did not speak the language of the users, or provided unusable error messages like “illegal input” or “oops, something went wrong”.
My research in the CUE-studies revealed that few people are doing orthodox heuristic evaluations. Instead they are doing expert reviews or persona based reviews, which my research shows can be highly useful. In most cases, people who claim that they are doing heuristic evaluations actually do expert reviews — or just reviews.
One of the recurring threads from your research is the need for common understanding across usability practitioners. What are your thoughts on the current state of the industry in this respect?
My experience is that usability practitioners do not always speak the same language. If you want to start an argument, ask questions like: What is the difference between user experience and usability? What’s the key difference between an interview and a usability test? What are user requirements used for? What does a usable user requirement look like?
That’s one of the key reasons why I started UXQB, the User Experience Qualification Board, together with 6 other people. UXQB provides a free, basic list of definitions of usability terms based on ISO standards with useful examples.
The curriculum and the associated certification tests are also useful for training our users: Development team members, marketing people, managers and other stakeholders, who do not work directly with usability but who want to understand what usability professionals do and what they can and can’t demand of us.
Up until recently, UX has been a discipline people moved into rather than trained in. However, there are now university courses, several accredited tests (both association related ones such as UXQB and BCS but also company created ones such as from Human Factors International and Nielsen Norman Group), and a deluge of immersion programs (General Assembly, Udemy and so on). There’s some concern from practitioners about this fragmentation, particularly in regards to the immersion courses. Any thoughts as to whether these concerns are justified?
I agree with some of these concerns. I am seriously concerned when I hear some of our competitors claim that after following their 5-day intensive UX course you will be a “UX Expert”. Statements like that banalise the UX profession and do not serve the interest of the UX community. The UXQB (and several of our esteemed competitors) do not claim that we turn out experts. Our modest claim is that after taking our foundation level certificate you will have the right to answer the question “Do you speak usability?” modestly affirmatively. Real experts are recognized as such by independent peers, not because they have bought a £2,000 certificate. Very roughly speaking it takes 2,000 to 10,000 hours and innumerable usability tests, interviews, observations and you-name-it to become an expert UX professional. I hasten to add that non-UX experts mostly do excellent work.
That said, I advocate certification. My experience is that training is taken more seriously if you have to pass a certification exam. But the exam is not the decisive thing to me. I run an introductory UX course with hundreds of computer science students at the University of Copenhagen each year. The course recently became mandatory, which I consider good progress. One of the statements from students that pleased me after my most recent course was “Rolf, you know, I will never work professionally with UX. But I am grateful that this course clearly proved to me that I am not a typical user of the products I make.”
One of the statements from students that pleased me after my most recent course was “Rolf, you know, I will never work professionally with UX. But I am grateful that this course clearly proved to me that I am not a typical user of the products I make.”
You were interviewed in Tomer Sharon’s It’s Our Research about stakeholder involvement. The UK’s Government Digital Services is now doing a lot of the things you mention such as ongoing testing of 5 people, the KJ method for collaborative analysis and generally getting stakeholders involved. Are there other places where you’ve seen this change start as well?
I am glad to hear that the UK’s Government Digital Services has adopted lean usability testing and involvement of stakeholders. I didn’t know that. Most organizations have considerable room for improvement in the usability engineering area. For example, they still need to understand that testing with 20 users provides only marginally better results than testing with 5 users. They still need to understand that stakeholder involvement is the key to success in usability testing — neither 400-page usability test reports nor one-way presentations of usability test results will do the job.
I believe that it’s one of our key duties as UX professionals to not just talk about usability but also to set a good example for our colleagues. For example: Do useful, interactive presentations! Write usable reports! “Sell” UX style guides to your developers instead of enforcing them! De-mystify what you are doing — it’s not rocket science!
Rolf will be keynoting at UX Scotland on his experiences of 32 years in usability. The conference runs from 8–10 June in Edinburgh, Scotland.
