Proper hammering with a microscope: Oversimplification in UX Research

Cornucopia: the horn of plenty. Copyright terms and license: Creative Commons.

I wish for each of the research methodologies in the researcher’s toolbox to be used once in a while when needed. At times it seems to me, that research is being reduced to practicing only one thing: hammering, by using one tool only, the usability test, a.k.a. the microscope.

Below there are thoughts on

• complexity and simplicity in product development, and the agility
• the rough draft of the researcher’s toolbox
• the actors who shape the understanding of UX Research

Warning: the article is one-sided, written from the perspective of a freelancing UX Researcher on short-term projects. I did give my best to try and execute it as close to objective as I possibly could manage but come on.

Definitions

I suggest that we shouldn’t strip the researcher persona of the richness of the tools at their disposal. A good usability test isn’t equal to good research, if what the product needs right now — is heuristic evaluation. Good research means selecting the most appropriate research tool for the current case in the given context. Of course, the researcher has to be able to conduct usability tests preferably with findings deeper than “users like” and “users dislike”. However, it seems that there are only a few tools that are gaining popularity, and becoming buzzwords, whereas the rest is gradually getting forgotten in practice.

How to distinguish good research from bad research.

Any research will bring lots of insights. Have you ever heard of research that uncovered nothing? Here is the meaning of the phrase “Any research is good”: You might not even know that the selected approach isn’t the best fit, because it will still bring at least a number of valuable insights. But here is the meaning of the phrase “Good research is better”: the selected research approach is the most capable of answering your current question combined with the appropriate use of resources: people, time, money. It’s the reason number one, why the expertise in complexity is beneficial for the researcher and the team they are on. Figuring out what works best requires practice, which means that failures are imminent.

Complex is simple

The second reason for the reduction of the research toolbox is balancing that border between the two natural processes that inevitably take place in product development and the course of life: complication and simplification.

The former is the one that has been happening since the Big Bang (or whatever other moment you consider for the start of our universe) — it’s the movement towards complexity, the cornucopia of everything. Brief clarification: in Greek, cornucopia means “the horn of plenty”.

The second process, simplifying, ordering, structuring, and cataloging — is the power of our human minds. It is not to be confused with wanting the world to be simple — that is not going to be achieved. If we are completely honest with ourselves, we also don’t need or want it. Simplicity does not exist outside of the human mind, so it is kind of like our superpower which enables us to grasp the chaos and operate in it.

As is true for every superpower, the wise and mindful use of it — is our decision and thus our responsibility. So, we are trying to reduce the number of tools since it seems simpler. This, however, makes the research a rather fragile concept. If we want to achieve some clarity at the end of any research cycle, why would we reduce the abundance of the means of getting there? To find simple answers, or to gain that clarity — we’d better dig into the complexity of our resources. So I say, let’s keep the fun of the plenty of possibilities in the field of research.

The more variety in the means there is — the more agility there is for the product in the ever-changing circumstances.

Since I have mentioned agility, I will elaborate on how the resourcefulness of the researcher supports this popular work frame, from my perspective.

On complexity in Agility

To BE it, it does not suffice to KNOW what it is and/or to change the NAME. It’s a culture. Some claim, agility doesn’t exist, meaning they haven’t encountered it. This statement is a hasty generalization. Agile in combination with good planning is working for many. Agile is the ability to acknowledge own mistakes and adjust accordingly.

What does complexity look like: the Research Tools

Here is what the Toolbox of UX Research includes but does not limit itself to:

• Usability Testing remote - in-person, moderated-unmoderated
• rapid prototyping
• card sorting
• heuristic evaluation
• expert evaluation
• survey
• diary study
• observation
• field study
• wizard
• work shadowing
• interviews
• questionnaires
• audits
• customer journeys,
• personas
• blueprints
• stakeholder maps
• contextual inquiry
• market analysis
• workshops (e.g. design studios)
• storyboards

Some of them seem like they include others or are part of another, and depending on the given context it’s true. Others might seem like annoying formalities and buzzwords. Seeing how uneven the possibilities are, it becomes clear why we tend to simplify. It seems like a way to speed up the process of development. However, I would like to point out here, that as an alone standing factor, the speed doesn’t matter. Clear concise strategy comes first. I once witnessed a presentation by a head of marketing that ended with: “If we aren’t the first, we lose.” Well, they still aren’t the first)

Whereas we think that the more we test the better research we do, in reality, which is also the big picture, the use of the same tool means getting better in the use of that particular tool. Only that tool is a microscope and it seems we don’t only look through its lens but also hammer nails with it, instead of simply taking a hammer from that toolbox. Now, as long as we touched on the very minimum curriculum of a UX Researcher offered by institutions, one cannot help but wonder if educational institutions are the reason for the complexity or simplicity of the research approach. Yes and no, the next paragraph is dedicated to the actors.

Who shapes the definition of the UX Research

In the mix we will find the following actors:

• me, the freelancing researcher. I was taking over the projects that literally were looking for someone to conduct and decipher the tests or interviews. And for the most part, there is an explicit request for the use of one of the big two: interviews or usability tests. I am a human being, so I like food and having a home. I take those projects, yes, and I am grateful for them. I am looking forward to more variety, more challenges though, and higher process effectiveness though.

Scene from the series “Friends”

• My customer. My customer, let us call them the PO, comes to me with the how. Whereas, they should approach me with the “what” — and save a couple of thousands and weeks of my customer’s resources ;) What’s money and time against prestige, right? You say “The results of usability tests show” — here’s an argument that’s hard to contradict. But really, try “These are the results of heuristic evaluation”. On the optimistic side, the meal’s taste depends a lot on the sauce you serve it under, right, so I would encourage the pitches for various tools no matter how lame.
• My customer’s customer. The people who use my customers’ products enjoy being a part of the conversation. The companies are very well aware of this, so usability research is a marketing tool. Some recruited test participants are happy to help out with their feedback regularly and make some money from it by registering on all recruiting platforms they can discover. So, everybody benefits from regular testing, or so it seems.
• UX Bootcamps. I was truly lucky to start in the field and to learn by doing. Several years later I graduated from a UX design boot camp. Bootcamp is where UX research takes up 1–1,5 lessons. They put a lot of effort into creating the most effective and rich curriculum for the given time constraints. However, the knowledge stays highly theoretical since it is oftentimes taught by people who just graduated from that same boot camp and/or have no experience in the field. There is a risk of getting the following impression of research: it’s there to justify my or my stakeholder’s solution. It’s true though that the teachers keep repeating the one thing: UX people never stop learning. Bootcamps help to start a new career with minimum time investment, people can change and live their lives thanks to them. However, due to the limitations in time, the complexity needs to be sacrificed.
• SaaS. To respond to the ever-growing demand for research, the agencies were founded. They don’t just offer their expertise, they go as far as developing specialized tools for research and selling the specific tasks that these tools were created to accommodate, for example, usability tests.

On the other hand, we have two agents who can give variety a chance:

• The educational institutions/ university degrees — they can teach the vastness and depth. Some graduates say their degree gave them 0. They might believe that, however, they are wonderful in what they do, they enrich the product development scene, and their life choices including the years of studies and getting a degree are the stepping stones to their success. Universities might encourage the training of critical thinking and stamina to stick out even with the most mind-numbingly complex and/or boring problems.
• The researcher on the contract. As it is true for every single point on this list of actors, the role is defined by an individual and therefore differs from one case to another. Surely, the researchers on the long-term contracts are meeting many limitations as well. Nevertheless, the extended exposure to the course of development of the same product enables them to learn and draw deeper conclusions. Ideally, they have time on average to play around with the various tools in the toolbox. But that depends on the UX culture in a company. I am talking rather about the potential here. I think the researchers on contract use the vastest tools due to the availability of resources. I didn’t want to leave them unnoticed, but they bring the most hope into UX Research, yay them.

These two latter points show that all good things require time.

Surely, there are more actors, but for the sake of bringing this article home this year, I believe this list should be enough.

Rounding up

Research has to be ethical, reasoned, transparent, clear (not the same thing), and fun. It’s a body in itself, let’s not rip its legs off. Preserve and nourish the complexity in UX Research by applying the most effective tools for the context.