Micro-Usability Test on TrueCar.com

Identifying the barriers for new buyers

What is TrueCar.com

TrueCar.com is an automotive pricing and information websites for new and used cars buyers and dealers. They aims to enhance the communication between consumers and dealers by providing accurate, timely and comprehensive pricing information. When buying a car, users can use their website to recognize a fair price based on what others actually paid and then reach Truecar certified dealers to ensure a better buying experience.

Executive Summary

Research question

How can TrueCar improve its experience for new customers? 
Focus on the price and both quality and quantity of the content.

Participants for an in-person micro-usability test

2 intended users who haven’t used truecar.com to purchase items before.

Where and when

Tested at users’ place or quiet room in the evening on January 8, 2017 Sunday.

4 major problems and solutions

  1. Not able to find new cars by price.
    Allow users to filter the options by price in both new cars and used cars.
  2. Not able to compare multiple choices.
    Add a comparison feature that is beneficial for users to gather a list of potential options so as to make decision more efficiently.
  3. A pop-up mask is annoying.
    Disable the default pop-up option. Allow users to trigger this function by themselves when they are ready to view more details.
  4. Target audience and current design is not aligned.
    Reconsider who are the target audience, so that Truecar owners can better adjust their interaction flow, information architecture and contents to fulfill users’ needs.

This article includes the details of how I conducted the research, the results I found, and a brief recommended changes.

This usability test report was written as a part of my graduate coursework in Needs Assessment and Usability Evaluation, at University of Michigan, Ann Arbor.

Research Goals

  1. Can users narrow down the buying options by price?
  2. Can users make buying decision from the information provided on TrueCar.com without referring other service?
  • Can they compare the option easily?
  • Does users use multiple services as assistance?

3. Can users estimate the price of selling an used car in the future?

4. What does the user think about TrueCar?

  • Do they buy into TrueCar service? Whether they would use the site in “real life”?
  • What do they think about TrueCar compared with other similar site that they have ever used?

Research Method

This micro-usability test is a fast and easy user research technique by doing a usability test with my friends. It’s designed to get almost immediate feedback on an interface with minimal overhead. After this quick testing, I will have a much better understanding of the customers and which parts of the product are difficult to use. It is also easier to set up compared with traditional usability tests.

Research Process

  1. Project planning: Clarity product’s goal, target population, including their needs and goals. Set up the objective, scope, timeline and test location of this test.
  2. Recruiting test participants from the target population.
  3. Designing the test: scenarios and interview questions
  4. Conduct the test
  5. Analysis results and propose solutions.

Screening Criteria for Participants

  • Users who are intended users
  • Should not have used truecar.com to purchase items before.
  • Have basic knowledge about how to choose a car
  • Who has budget concerns
  • Who know well about the brands

For this micro-test, the recruiting method I used was reaching out my friends who fulfill the screening criteria.

Participant Profiles

Participant 01

  • Male, expert user who like cars very much, including brand, series, wording, advanced information, general cost of owning a car.
  • Already bought a car within a year.
  • Student who has a more flexible budget range.

Participant 02

  • Male, normal users who have basic understanding of famous brand knowledge.
  • Already bought a car within a year.
  • Student who has budget concerns and don’t care luxury stuff.

Preamble for the Test

“You’ve been invited to help us understand which parts of the product work for you and which are confusing. Truecar is a website that you can find a car with detailed pricing information and allows you to contact the dealers through this website. Even though we’re calling this activity a usability test, you’re not being tested.The product is.There’s nothing you can do wrong. It’s not your fault if you can’t get something to work, and you won’t hurt anyone’s feelings if you say something bad about the product. It’s really important that you speak all of your thoughts aloud. Think of it like a “play-by-play” description of what you’re doing and why you’re doing it. I’ll stay in the same room and quietly take notes, but you should ignore me. Just focus on describing what you’re doing.”

Citation: Goodman, E., Kuniavsky, M., Goodman, A. M. (2012). Observing the User Experience. A Practitioner’s Guide to User Research 2nd Edition. Morgan Kaufmann

Test Scenarios

Partially assigned by instructors

  1. Your budget is $17–20000, you are concerned about ensuring that the vehicle performs well in cold weather. Please send it out to me who you are jointly purchasing the vehicle.
  2. Your friend would like to be able to sell the vehicle in a few years when you both graduate and want to be able to have a rough idea of the cost of ownership.
  3. Your friend prefers japanese brand, because the maintenance fee is lower.

Post-Test Questions

  • What’s your general impression? How would you describe this product to someone?
  • What was your favorite aspect of the product?
  • What was the most confusing part of the test?
  • Would you continue using this product?
  • Would you recommend this product to a friend or colleague?

Hypothesis Before the Test

I conducted a quick heuristic review before the test to identify the issues users might face in the test. If users act the same as my assumption, I can dive into the reasons behind it. If they act differently, that would be an interesting finding and a good chance to learn how users think differently.

Potential Problems

  • Price: hard to filter by price.
  • Domain knowledge: difficult terminology.
  • Flow: popup is annoying.

Key Findings and Solutions

Generally, the participants feel frustrated during the test. Truecar emphasize the informative pricing information, however, participants who has a limited budget didn’t find a simple and efficient way to filter options by price. They needed to keep changing pages and use additional tools such as screenshot or writing down on paper to collection the possible options. In additions, they felt that some interaction design is annoying.

Problem 1: Not able to find new cars by price.

The severity of the problem: high

As mentioned before, users can’t not find an efficient way to choose a car based on price range. Participant of this test used their own understanding of brand and models to guess the price before entering the detailed page of the car, hence, sometimes, they felt frustrated and wasting time when their guess are wrong.

It is confusing that only “used car” can be browsed by price. — User1
Solutions
Allow users to filter the options by price in both new cars and used cars.

Problem 2: Not able to compare multiple choices.

The severity of the problem: medium

Both participant can’t find the function that assist them to make a comparison between several options, therefore, they noted down the option by themselves.

For new cars, it’s fine for me to compare just a few options, however, for used cars, I need a advanced comparing features to help me with trivial details. — User2
Solutions
Add a comparison feature. Before making a costly decision like buying a car, people will compare several options. Therefore, a comparing feature is beneficial for users to gather a list of potential options so as to make decision more efficiently.

Problem 3: A pop-up mask Is annoying.

The severity of the problem: medium

When entering a detailed page of a car, the pop-up mask appeared to assist users to choose advanced option of this car. However, at the very beginning, users just want to gain a quick idea about what are the possible options without too many details. Hence, they were interrupted by the pop-up and got tired of kept closing it.

Solutions
Disable the default pop-up option. Allow users to trigger this function by themselves when they are ready to view more details. For example, design a button with a fascinating marketing copy like “Get your best plan within 60 seconds”.

Problem 4: Target audience and current design is not aligned.

The severity of the problem: high

At first, the assumption is that targeted users are who already know a lot about brand, models and other professional wording, because current design required users to choose cars according to brand, models. Novice may not recognize the brand, models, levels, package, etc.

In this user testing, intermediate only referred the basic information. He quickly glanced the basic information and the gallery to see the styles. He said that he will not read the detailed information and advanced specification, because he couldn’t understand the professional terminology. While testing with expert users, he said that the information is not detailed enough for him to reference or the details are ignored because they are hidden. Hence, normal users can not understand the detailed information provided on TrueCar while experts need more advanced information.

Solutions
Reconsider who are the target audience, what are required background knowledge to use TrueCar. After redefining, TrueCar owners can better adjust their interaction flow, information architecture and contents to fulfill users’ needs.

Other Problems

The severity of the problem: minor

  1. Detailed Information is different across brand.
  2. When using search function for used car, the search criterias will lost after users changed to different page. In additions, users can’t set multiple options for single criteria.
  3. No information about cost of ownership.
  4. Users are required to sign up to see more information, however users can see details without an account on competitors’ website.

What Works Well

Although, there are several places to be improved, users acknowledge the informative pricing information about what is a fair price. They can refer this price before reaching dealers so as to get a better deal. In additions, the price trending help users to decide which seasons is the best time to buy a car. For instance, if they find that summer vacation is cheaper, they may decide to buy it in that cheaper time.


Recap

4 major problems and solutions

  1. Not able to find new cars by price.
    Allow users to filter the options by price in both new cars and used cars.
  2. Not able to compare multiple choices.
    Add a comparison feature that is beneficial for users to gather a list of potential options so as to make decision more efficiently.
  3. A pop-up mask is annoying.
    Disable the default pop-up option. Allow users to trigger this function by themselves when they are ready to view more details.
  4. Target audience and current design is not aligned.
    Reconsider who are the target audience, so that Truecar owners can better adjust their interaction flow, information architecture and contents to fulfill users’ needs.

Discussion

Limitation of this test

Since this is a micro-test, only 2 participants are recruited. This limited size of testers may not able to cover the issues in a comprehensive view, So further research may be required to further verify the credibility. In additions, recruiting users from friends may also increase the bias due to similar background and living experience.

Next step

Since I don’t have a valid login account, so the tasks scenario may be limited. With a verify account, I would be able to test completely different functions of the site — such as finding a local dealership and scheduling a test drive.


Disclaimer: I do not work for or represent TrueCar.com. I’m just a passionate and curious UX researcher and designer, who loves talking to people and solving problems . I’m happy to hear your feedback on this project and to chat more about other ideas. You can also reach me at galeyang.com

Show your support

Clapping shows how much you appreciated Gale Yang’s story.