Data-Driven Design
In the ambiguous space of product development, where we deal with uncertainty in various ways about users and business, the use of data can act like a light to follow this path for us as designers, with less possible failures and also more learning and more success.
Before talking about “Data-driven Design”, we must first be familiar with “design”, “problem”, “data” and “information” concepts. Because in the following we need to know these keywords and their relationships with each other. There are several articles, papers and books about each of these keywords that talk about them in detail, but here we briefly review the definition of these keywords.
Design: Creative problem solving; In fact, we use two halves of our minds in design. It means using logical thinking along with creative thinking, so that we can provide solutions for various problems and these solutions are creative and innovative as well.
Problem: Ian Robertson says in the book “Problem Solving” that when we know the current situation, we also know the target situation, but we do not know the way of getting from the current situation to the desired one, so a problem arises. In fact, the gap between the current situation and the target situation of a process or a product is called a problem.
Data: Propositions or raw facts that have not been processed and no special meaning has been extracted from them.
Information: Data that have been organized and processed and meaning extracted from them.
What gets measured gets managed. The numbers tell the story. Peter Drucker
Data-driven Design
Data-driven design is a decision-making approach to the design process;It means to identify the problem, to clarify the problem, to start solving the problem and to measure the effects and results of our actions through the help of data and to repeat and improve this process based on it. Using data, we realize whether we need “iteration”?! Or the data tells us that this process does not need to develop and iterate at the moment!
Some problem solving methods in design, including design thinking, user-centered design, and double diamond, are also data-driven.
The Importance of Using Data in Design
Without data, we walk through the product design and development path blindfolded. The product development process is a little different in practice from the burnished and clear path described in some books and training courses. Many parameters such as time, human resources, technical skills, costs, the business growth stage etc. may change the priority of the actions and in this ambiguous and tricky space, the combination of data and design can create an “excellent value cycle” between users and the business.
Collecting, managing and analyzing data creates a connection between design, user experience and business needs. This data can be gathered in different ways, from different sources and by using different tools. But the important point is to know which data set to use and which to ignore in each design.
Advantages of Using Data in the Design Process
- perception of the whys the wherefores of actions
- perception of actions results
- Saving time!
- design in line with the best practice
- perception of failure and success
- Efficient and effective design
- using data to direct innovation
- increasing the solutions reliability
- help trust facilitating and support of stakeholder
When we design with the aid of data, we know what goal we are going to achieve by defining metrics, and by using data, we can also monitor the success or failure in achieving the goals. We can also monitor the probability of success or failure in achieving the goals. When we clarify the problem with the help of data, our ideas and innovations are presented in an appropriate framework, and because we have more reliance on and more command of the problem, we can choose best solutions for our problem more easily. In addition, stakeholders can trust these actions better in the design process when they see our theories and actions based on data.
Have you ever used data in the design process?
Well, the answer to this question is “yes” for a great many people. If you do interviews in the design process, use questionnaires, monitor reports received by the support and PR team, create events and check Google Analytics or Firebase, monitor news sources and media about your product, conduct usability testing, read different documents to observe the standard distance between icons, do a case study, consider a feedback recording component for your product, monitor churn, do A/B or A/B/C testing, monitor acquisition, monitor retention, analyze all of them, get a preference test, have groups for users feedback, do card sorting, create user journey map etc. All these show that you develop your design with the help of data, and these actions occur in different stages of the design; But we cannot name our design process a “Data-driven Design” just because of the items mentioned! I will explain what data-driven design is.
Different Layers of Using Data in Design
- Data Driven Design
- Data Informed Design
- Data Aware Design
Data Driven: According to King, Churchill and Tan, “Data Driven Design” refers to making design decisions based solely on quantitative data. In this framework, data is of primary importance.
When the main goal of the product is to optimize performance, this approach can be efficient.
“Data Driven” design means that the collected data determines the design decisions. Sometimes, we can definitively answer different questions that the product development team asks by collecting data from various tests and as a result, make the best design decision.
In some cases, this would be the right way; but not always! In fact, if we know exactly where the problem is, exactly what the goal is and have a very precise and unambiguous question, we can use Data-driven.
As a simple example, we can refer to an A/B test, in which the final decision is usually made based on that. It is important that the methodology and measurements must be correct, and also the type of question we want to answer is one that the (quantitative) data can answer. This relies on a precise understanding of the types of chasms that data can bring, and I will provide a further explanation about these chasms.
Data are often systematically biased, and limiting decision to only what we can currently measure often de-prioritizes larger, more important aspects of the problem and increases the risk of action. There are some limitations of using this approach in organizations; for example, if the required infrastructure, knowledge and expertise in this field are available, and it should be among the managers’ strategies and attitudes so that they can deal with its cost in terms of time, human resources, energy, etc.
On the whole, the Data Driven approach answers meaningful questions; where data alone can lead to decision making.
Data Informed: “Data Informed” design is an approach which is more flexible in using data in the design process. In this case, additional factors like qualitative information, expertise, instinct and experience can be prioritized as well as quantitative data.
In some cases, the design decisions may have many details or we may not know exactly what we should achieve and there is no clear question; In these cases, making decisions based solely on data does not lead us to the right answer. This is where we use the Data-Informed Design approach; and a product development team and of course a designer, consider quantitative data as only one of the effective factors in the decision-making process. In these cases, we need more research, collect different types of qualitative data, rely on experience or creativity and innovation assistance. For example, we use this approach in the process of discovering the needs of users and providing a value proposition.
Thus, in Data-informed design, we may not be fully purposeful and well-informed about what we want to understand.
Decision making in this approach requires that we have systematic thinking, be aware of our biases, and our approach is to act and learn quickly. This decision-making is a little more creative, broader, and of course requires more iteration of the process to come up with the desired solution.
It is easy to fall into the trap of biases and unknown assumptions when we do not have a systematic approach to data analysis.
Data Aware: In this approach, data informs us of an event; we put quantitative data on the same footing as other decision-making factors, and in fact, in this approach, they merely inform us of “what happened”. In Data Aware design, creativity can happen more prominently and we realize that design decisions are not always made based on data, but ideation, creativity, innovation, experience, and creating hypotheses can also be involved in this decision making.
Superior Approach
The superiority of the three proposed approaches depends on some factors such as product development strategy, data infrastructure, the issue we are going to address, meaning whether it is problem discovery? problem solving? New feature development? And various other factors, and by default, none of these three approaches is superior to the others.
Authentic case studies are good sources to examine the data-driven design process to see what achievements this approach has represented for products. For example, on the “Growth Design” website, we can see good case studies which particularly offer explanations about some of the digital products. I also wrote about the actions that were taken as a result of the analysis of “users’ behavioral data” in the article about the first impression on the platforms. It means the closer Time To Active of the users to his first login, the more likely he will return to the product in the next month.
Data types
Data science experts name and classify data in different ways according to different data uses. One of the types of data classification, which is helpful in product design and creates a good attitude for the designer, is presented in the book “The Lean Product Playbook”, and I will explain this classification.
Quantitative Data
numerical data that provide information about “who”, “what”, “when” and “where”. Quantitative data show the scale and do not say anything about the cause of the issues. The bounce rate on a website is an example of quantitative data.
Qualitative Data
data which inform about “whys” and “hows” of events. For example, interviewing users to discover why something happened is qualitative data.
Behavioral Data
data collected through monitoring the behavior and actions of users. For instance, a field study about a subject is a type of behavioral data.
Attitudinal Data
The data gathered through the opinions and words of users. Asking users about whether they use a certain feature or not is a simple example of attitudinal data.
Combination of the four types of data mentioned above leads us to the following matrix:
- Qualitative- Attitudinal
- Qualitative- Behavioral
- Quantitative- Attitudinal
- Quantitative- Behavioral
About some examples of the matrix above, I’d like to add that depending on the method of data collection, they can be placed in another category. For example, through usability testing, we can obtain “quantitative-behavioral” data in addition to “qualitative-behavioral” data; or through a structured interviewing, we can also obtain “quantitative-attitudinal” data.
It’s important that none of the types of data are trivial and each of them is collected and then used in their own cases. Sometimes we get the answer to our question with just one type of data and sometimes we need to collect and combine them. I remember in one of the interviews, one of the applicants who had a good experience in this field expressed that the data obtained through the interview becomes obsolete and no longer useful! Well, that’s not true and we use a variety of interviews and qualitative data collection methods to understand the reasons and deeply the users’ needs and commercial stakeholders.
Generally, data is not just numbers. Qualitative data, which refer to items like feelings, opinions, and observations, are data while they cannot be expressed numerically. Both types of data are valuable because they are complementary.
The qualitative-quantitative issue is a really misunderstood area of research, especially for people who haven’t been exposed to expanded education. Dave Yates, Senior UX Designer at Bazaarvoice
Bates says: I have encountered many cases where people ignore qualitative research because they don’t understand that non-numerical data is still data.
Hiring a world-class designer is not enough to guarantee a product’s success. Designers cannot predict what users want and what they need.
Jared Spool is the designer of the “300 million-dollar button” and the owner of the famous quote “Good design, when it’s done well, becomes invisible. Think of it like a room’s air conditioning. We only notice it when it’s too hot, too cold, making too much noise, or the unit is dripping on us.” says:
Data science is now an essential skill for every UX team. If you don’t have people who understand how to do data science, you cannot create great designs.
Chasm of Data-driven Design
- Collecting the Wrong Data and Wrong Data Collecting: Collecting the data with the wrong methods and tools, or collecting data that we do not know the purpose of collecting, ultimately leads to our analysis and decision-making being based on the wrong data and we rely on this or put our money and energy on data that we don’t know for what purpose we collected it, and we will face “data waste”.
- Being Captive to Data: creating a balance between intuition, expertise and experience is a matter that makes us not too dependent on data and not to stray from innovation, which is considered one of the parameters of a good design.
- Haste: We need time in order to obtain data and then analyze it, and sometimes by rushing in this direction, we collect data with poor quality and reliability.
Data quality
Quality is an important criterion which is defined based on parameters such as accuracy, integrity, compatibility, reliability, being up-to-date and timeliness. Although data engineers consider the principles and standards related to it, this criterion should be taken into account in the data collected by the designers themselves and there should be a great deal of tact towards data being of standard. As a whole,various products and tools have made collecting and accessing data easier than the past, and what is challenging is the quality and analysis of the data.
Designers + Engineers, Scientists and Data Analysts
Data is a vast and complex field and we need data science experts to collect, analyze and reach conclusions. The close interaction between product and user experience designers and these experts helps us step more confidently on the data-based design path, achieve the benefits of this approach, and avoid possible chasms.
Sources
- Designing with Data
- Lean Product
- Springboard
- Motamem
Special thanks to Mr. Omid Mirabzadeh and Ms. Shakiba Tashrofi.
