What is Benchmarking?
UX Benchmarking evaluates user experience by measuring specific metrics and comparing it to similar products in the market. This helps assess product design progression, market positioning, and competitiveness relative to competitors.
A benchmark is a comparison of a product to other selected products based on meaningful relationships established using a constant. The combination of these selected products and relationships creates a benchmark space. This space can be visualised as a circle with the product in the centre, surrounded by circles representing competitors and network structures showing relationships between them. The defined benchmark space holds all products and relationships together and facilitates idea generation, improved visualisation of work, increased work dominance, and more productive results.
If I call you ‘colour’ and we look out the window, we look meaninglessly out of the window, but if I call you red and look out the window, we start to catch red colours, maybe we notice a red-coloured object that we have never noticed before, and maybe the object’s redness has nothing to do with its surroundings. We begin to think about the relationship. Our brain, which is lost in the infinite universe with the word colour, becomes defined with the word red, creates a space for itself and is triggered to think. In fact, this is how the benchmark study is defined.
Defining the work allows us to have a flexible space, and when we are flexible in design, we get more creative results. For example, I will do a benchmark study for an e-commerce site project. Limiting my work to examining similar, competing and pioneering firms weakens the flexibility of the space. But besides these, nurturing my work with topics such as different sectors and new developments in technology stretches my space and increases the diversity of ideas by obtaining more data. Reaching different topics will be formed by defining the areas around my existing benchmark area. Defining the surroundings of my benchmark space is also related to the definition of my own space. “Benchmark’s 5W1H” will guide us to define the benchmark space.
Benchmark is the comparison of a product with other selected products by establishing meaningful relationships according to a determined constant.
5W1H for Benchmarking
The concept of benchmarking can be done consciously or unconsciously in the design process. When receiving a brief from a client, we start by researching similar products to understand what kind of product we will create. This is a benchmark. As the product becomes more defined, we continue to research and compare similar details to further understand the product. This is also a form of benchmarking. By being aware of the benchmarking process, our project will progress more efficiently and ultimately be completed more efficiently.
To guide our research process and increase our awareness of benchmarking, we use the Benchmark’s 5W1H structure, which consists of 6 questions:
Who are we comparing? We determine the sectors to be compared based on the dynamics of the project, such as competing sectors, similar sectors, and leading sectors.
Why are we comparing? This question is related to the project brief and asks what the purpose of the project is.
What are we comparing? We determine what is actually being compared, for example, the homepage of a company.
Where do we compare? We can use various tools for benchmarking, such as an excel or word file, or Miro Board, which is a useful tool that allows for visual and textual content to be easily organised.
When do we compare? The time frame for comparison is customised based on the project deadline or a specific time specified.
How do we compare? This is customised for the project and based on the working method of the person conducting the research. For example, if comparing the homepage of two brands, the researcher could create two rows, one for each brand, with columns for mobile and desktop. They could then add images of the homepage and take notes on areas of interest. The structure and methods used for benchmarking can vary depending on the dynamics of the project and the working method of the researcher.
Why Benchmark?
Who am I?
Let’s pretend we are an office chair and trying to figure out which type of office chair we are. To do this, we start by researching different types of chairs, such as beach chairs, lounge chairs, office chairs, and dental chairs. After realising that we are an office chair, we delve deeper into researching office chairs and come across many different types. To determine which type we are, we examine our features and compare ourselves to similar products. This process of self-discovery and exploration helps us understand our strengths and what we can add to improve ourselves. Ultimately, it strengthens the definition of ourselves and our surroundings.
Observation
Observing is a crucial aspect of benchmarking, as it helps us understand the usage of our product and user behaviour. A well-known example in UX is Norman Doors, where Don Norman noticed a problem with user behaviour while observing his friend using pull-or-push doors. The user struggled to open the doors as they couldn’t determine whether to push or pull due to handles on both sides. Norman realised that the form of the product could solve the issue, rather than writing ‘Push’ and ‘Pull’ labels. Observing user behaviour in this case uncovered usage problems. To summarise, observation is vital in benchmarking and helps identify both positive and negative features.
Inspiration
Henry Ford revolutionised the automotive industry in 1912 by introducing the assembly line system during a visit to a slaughterhouse. This example showcases how benchmarking can inspire new ideas and influence different disciplines. Benchmarking can be applied across various industries, disciplines, and product groups. By exploring diverse examples, we can expand our benchmark space and find new sources of inspiration. By examining the areas surrounding the benchmark space, we can generate more innovative ideas, as we mentioned earlier when discussing the concept of a flexible space.
Mentor
By identifying leading companies as masters in our benchmarking study, we can learn from their example and understand the path they took. This process, which could be described as “the apprentice who follows the master finds his own way” will broaden our perspectives and help us develop our own methods. Emulating the practices of masters will guide us in charting our own course.
Good Points / Bad Points
When making comparisons, we establish relationships between the examples by comparing the ones we have analysed. This allows us to identify both the strengths and weaknesses of each example and determine what to focus on. For instance, if we consider two e-commerce sites, we may notice that their search bars are different and consider which one is more beneficial. To understand the impact of this difference, we could research relevant studies to gain insights and understand the importance of the search bar for e-commerce sites through the data we collect.
Conclusion
A benchmark study is crucial in obtaining reliable data. For this, it’s crucial to have a clear understanding of what a benchmark is. To make a benchmark study more structured and defined, the 5N1H framework can be utilised as a guide.
Thank you
