Think about ChatGPT limitations before you use it for Design Research

Recently, I came across several posts discussing the usage of ChatGPT in design research, particularly in creating design frameworks like customer journeys. While ChatGPT is a powerful tool driven by AI technology that can provide human-like responses to your queries, it has limitations that researchers should be aware of. In this post, I will share my experience and thoughts on where ChatGPT can be useful in research and where it may fall short.

What is ChatGPT?

ChatGPT is a natural language processing tool that can summarize vast amounts of data in its database. Its database is updated as of 2021 and contains 570 GB of data, but it is not online, so it does not have information on recent events or invitations. While ChatGPT can provide sophisticated answers to your questions, it has limitations that researchers need to be aware of.

ChatGPT for secondary research

ChatGPT can answer your questions very sophistically, but you must fact-check them. ChatGPT answers can contain hallucinations, which means they look logical, but ChatGPT made them up and are not accurate or correct. Also, it made up references when you ask for them! Believe me, I tried. I asked ChatGPT to give me resources to read about design thinking and its impact on customer experience. As you see, it provided me lists of links that look legit, but for God’s sake, non of them are real. So don’t let it fool you.

I tried all of these links, and none of them are real.

Some of the links from the previous image

Despite this experience, ChatGPT can be a valuable resource for secondary research. We only need to learn its limitations and find the best practice to use it.

In this example, I asked ChatGPT about the main complaints regarding US federal government services. The response is quite acceptable; there is nothing novel there for a person with experience in this subject. It is good for a new researcher in this field, although this data should not be accepted blindly and must be vetted.

ChatGPT for primary research

Primary research is the foundation of any innovation. This is the most critical part of the process that gives the team an understanding of what question(s) they must answer.

They create data that had never existed before so that no AI can help. On paper, ChatGPT can produce artifacts such as customer journeys, but it doesn’t provide any new valuable information for the team.

I asked it to create two customer journeys for obtaining a driving license, one for an ordinary person and one for an international student in a rural area. As you see in the screenshots, the result is so generic, failing to identify pain points.

A comparison of two customer journeys created by ChatGPT: one for a US citizen on the left side, and another for an international student on the right side.

For example, it missed the classic catch-22 situation of the road test. For those who do not know about it, you should bring your own vehicle for the road test in the US. This is challenging for people who don’t have someone to borrow a car for the test, like international students. Legally, people can rent a rental car for the test, but:

1. Driving license is required for renting a car.
2. Most rental companies ban users from using their cars for the test.
3. Some rental car companies do not rent cars to people younger than 22 yrs old, even if they have a license (like an international driving license)
4. Not every people can effort renting a car.
5. Rental companies are not available in all communities.
6. Credit card is required to rent a car. Many young people, especially international students or students from low-income families, do not have credit cards.

I am not an expert in this topic; I am only an ordinary person who went through this process, yet I know these issues. I also heard many stories in my community, from first-generation Iranian immigrants, about how many officers in rural areas caused problems and discriminated students due to the lack of law knowledge about laws. You want to uncover these types of data through primary research. Without this understanding, the design would be reduced to cosmetic improvement, not a fundamental change to the user experience.

Wrap-up

While ChatGPT can bring value to the design process, it is crucial to consider its impact on the most vulnerable part of society before using new technologies. AI can deepen our biases because it is built upon our biases. AI can put us in an echo chamber and disenfranchise people already suffering from the system.

Therefore, researchers must use ChatGPT cautiously and consider its limitations and biases before relying on its responses. Here are some of my recommendations:

Do’s:

• It can be utilized as a conversational search engine, providing a potent means of obtaining historical information and context on any topic that occurred before the year 2021.
• ChatGPT is capable of generating lists of references to help you learn more about a topic. Interestingly, if you ask ChatGPT to provide a list of books or blogs on a specific subject, it can do so. However, if you first ask ChatGPT about a topic and then request a list of references to what it has already said, it fails to generate a satisfactory response.
• ChatGPT is an excellent writing assistant and poses a significant challenge to Grammarly’s business model. It can enhance your text by editing it to be more formal, friendly, or even simplify it to a level suitable for a 4th grader. Full disclaimer, I used it for this post.
• By providing keywords, ChatGPT can generate a statement for you. It is especially useful for creating artifacts such as personas. However, it is essential to note that you must supply the necessary data and avoid relying on ChatGPT’s data.

Don’ts:

• It is crucial to fact-check ChatGPT’s data before relying on it. It is similar to reading a blog post — if you stumble upon an intriguing piece of information on a blog, it is wise to verify it before accepting it as true.
• It is not advisable to rely solely on AI to replace secondary research. AI is inherently biased, lacks empathy, and can only analyze existing data. As a researcher, it is your responsibility to identify unmet needs and collect relevant data.
• Avoid requesting ChatGPT to generate design artifacts or fill frameworks solely based on its own data. The information provided may not be current, and there is a risk of inaccuracy.

I have shared my personal experiences with this new tool and I believe that it will continue to improve over time. I am eager to hear about your thoughts on this post and your own experiences with ChatGPT. Please feel free to share your feedback and insights in the comment section below.