Enterprise User Research — Part 2
In this article, I am evaluating some of the known techniques as a source of relevant user feedback to be used in product design practice. They are all important tools in engaging with users and customers. But they are not always providing the right data to be used in a specific domain of professional software design.
- Interviews
- Testing and validation
- Surveys
In Part 1 of this article I am summarizing the overall challenge of professional user feedback.
Interviews
User research is not one-time-only activity. Unfortunately in fast and iterative agile environments, there is often no time for repetitive ethnographic research.
Observational user research is often completely skipped. But even when not skipped completely, this user research is not executed by professional researchers or designers.
Research interviews are led by "SME's" domain experts, usually pre-sales or customer services consultants. These experts don't have a deep understanding of user research techniques. Results of this "research" are only confirmations of "what we already know" or asking "What do you want" type of questions.
When pointing to this fact, I was once asked if I am positing that only design professionals can execute a valid user research.
We are all humans and nobody is perfect. But a design professional is not only aware of these biases, but he is also trained and skilled in mitigating the risk of false results.
For distributed teams, it is simply not possible to execute all the research in-house. When research objectives are clear, I believe that it is good practice to outsource user research to remove any unintentional biases.
Companies definitely know their current customers’ wants, and their desires for the next version of the software. They generally don’t have a good understanding of what their customers truly need, and are expecting to do with their software. If you don’t have a user researcher on staff, or if you’ve never used a researcher before, you just have too much-ingrained knowledge. Even when you go and ask questions of your current customers or put surveys together, you’re inserting so much bias in the way you ask questions that you don’t really get the data we need to come in and build a really good user experience.
From You can’t build a good enterprise UX without user research by Jon Reed.
Testing and validation
The more limiting is the input from user research, the more important is to validate every design assumption. This is another specific challenge of enterprise UX design. For a consumer product, the designer can have a pool of hundreds or thousands of participants to choose from. But for a professional app, there might be only a few keen users. And these users are always the same over the course of years.
Internal stakeholders also often fall into frequent user testing and validation misconception trap. They are confusing product presentation with follow-up discussion and brainstorming with a structured user testing and validation.
I found remote prototype validation to be the key user testing technique that worked for me. I am still exploring various tools and approaches, but I see huge potential in this area.
When performing unmoderated user testing, you are always risking biased results of self-reported task completion rates. That is one of the reasons that whenever possible, I always prefer moderated remote user testing and prototype validation.
Until you put a prototype of a new feature in front of a customer, there is NO WAY to determine whether it meets the customer’s requirements to actually use it.
From The Value of User testing by Steven Cohn | JustInMind.
Surveys
Surveys are very popular at corporations and I am also guilty of creating a few surveys on my own. Most of the time, corporate surveys just represents lazy design — they are boring and poorly crafted.
Corporate surveys are not crafted by usability professionals, and they are just asking questions where the desired answer is obvious.
See how Wallmart leading survey wording cost the company over a billion dollars.
It is also tempting to combine surveys with some existing standards in UX benchmarking such as SUS or NPS. The most recent corporate survey I saw started with "…this VERY simple 20 question survey…" followed by a mixture of SUS, NPS and random leading "How do you like X" questions.
Read this great summary on Net Promoter monster by Matt LeMay.
My main lesson from surveying is that surveys are hard. It is not about throwing some questions into SurveyMonkey. The effort invested into really good and professional survey might be not worth the benefits. Simply because surveys will not provide qualitative background about motivations (Why one answer over another?) and the number of active participants will not form a statistically significant sample to consider quantitative results.
Conclusions
I don't have fixed set of recommendations about what user research techniques works best in an enterprise environment. I am still trying to develop my own set of techniques and hope to continue in this article series.
I have the best results with remote user research and early prototype validations.
I am still working on introducing new user research techniques. When it is not possible to directly observe users performing the task, I am thinking of introducing diary studies to record tasks offline. But because of security and legal constraints, this represents another huge challenge in an enterprise environment.
I would love to perform at least few studies with the Eye-tracking camera, especially with respect to complex data visualization and iconography of professional apps.
Another research and validation technique that I want to explore more is telemetry and data analytics. For on-premise enterprise apps, this represents another massive challenge to tackle. But nothing is impossible and I look forward to extend my research and validation techniques in near future.
If you liked the article, please recommend it or respond with a comment.
