Initial Usability Testing of the Ecosystem SDK

Usability testing is a user testing technique that helps us:

• See how users interact with our product
• Uncover any pain points in the design
• Discover any areas of opportunity that may exist where we can provide a better experience to our users

By conducting usability testing early in the development process for our Kin Ecosystem SDK, we can see how users interact with different, potential versions of our product. This allows us to to better pinpoint what the user experience should actually be.

The Ecosystem team had a couple of different approaches in mind for the launch of the first Kin experiences for users within a messaging app. To test their effectiveness, we conducted usability testing with young adults (18–20s year old) who have shopped online within the last 6 months and use a messaging application.

The main objectives of these usability sessions were to obtain insights on:

1. How can we make the on-boarding clear to users who may be familiar with Kin and those who may not?
2. How can we make it clear which areas in the marketplace are “earn” and which areas “spend”?
3. Do our current earn and spend user flows make sense to users?
4. Is the transaction screen as simple as it can be?

OBJECTIVE 1: On-boarding Usability Testing

Going into testing, we had two possible versions of an on-boarding experience. To understand which features of the versions work best, we assigned some users to condition A and others to condition B.

As the screenshots show, participants in Condition A entered the Marketplace as soon as they pressed the Kin Marketplace icon in the message app interface. In the Marketplace, the only available activity to complete would be a “Let’s Get Started” Kin activity. When selected, users would see a one-page explanation of Kin. After they pressed “Let’s get started” they would get 6,000 Kin and then see the rest of the potential earn/spend opportunities in the marketplace.

In Condition B, participants would see the Kin information page and receive 6,000 Kin before even entering the Marketplace for the first time. Once they were in the marketplace, they would immediately see the different earn/spend options available.

Note: These screens and details in them were used for research purposes only so that participants can understand the marketplace.

Task: We asked users to complete certain tasks and answer questions about what they thought what they could do in the marketplace. From their interactions and responses, we observed how well they completed our set tasks without guidance. We also sought to understand how confident users felt using the app, what they understood of Kin, how quickly their learning curve was, and their overall impressions.

Through testing we saw that

1. In Condition A, participants weren’t confident about what to expect upon entering the app. They were unsure about what the marketplace was and relied on the UI to contextualize and infer what this experience could be. Since participants in these sessions are partaking in a study, they did take the time to understand the marketplace — however, a first time user in the real world may experience confusion and not take the time to understand the marketplace itself and may instead just leave.
2. Those that flowed through Condition B were more confident in describing what was going on in the Marketplace soon after having entered it for the first time.
3. In both conditions, many people skimmed the Kin information page, but were able to grasp the key concepts before moving on. However, those in Condition B weren’t sure why they got 6500 kin at the start and voiced that they would prefer it if the screen itself told them explicitly that they would get kin by pressing that button.

Outcome: In the end, we landed on a hybrid solution: Now, users see a splash page before entering the Marketplace and then complete a short tutorial as the first earn task that provides them with key information on the marketplace and how to use it.

OBJECTIVES 2 & 3: Testing on the Earn and Spend UI and User Flow

After users in Condition A & B were on-boarded onto the experience, we looked at what questions and pain points arose as they completed earn and spend opportunities.

Tasks: We asked users to find and complete an earn opportunity, and then asked them to complete a spend opportunity.

Through testing we saw that:

While users were able to complete both tasks successfully, a couple of areas in the spend section caused some confusion.

• In a scenario where users accidentally pressed a spend opportunity, it was not clear to users that they could exit the experience without making a purchase. An ‘X’ was added to the screen to clarify this point.
• Some users found the purchase flow too quick and abrupt. We adjusted the confirmation button in response to the feedback, adding “Confirm Purchase” so that all users would understand that pressing the button completed the process.

Note: The details presented on these screens were intended for research purposes only and are not final.

Outcome: Copy is key! Our testing made it clear that we need to provide users with better information regarding how to redeem, where to do so, and where they can access their purchased code.

OBJECTIVE 4: Testing the Transaction Screen

Our transaction screen needs to provide users with information on their previous Kin activities in the quickest and simplest way possible.

To make sure we achieve that, our user testing included questions about the Transaction History page. We used these questions to validate our direction and ensure our approach to presenting transaction activity that makes sense to users.

Note: The details presented on these screens were intended for research purposes only and are not final.

Task: Users were presented with a screen that did not include descriptors — just the Kin earn/spent amount only. We then asked testers to indicate which rows reflected completed earn opportunities and which reflected completed spend opportunities.

Through testing we saw that: While users were readily able to identify which rows were earn versus spend, they primarily relied on the “-” icon next to the 8,000 to be confident that that row reflected a spend opportunity and deduced that the others (because they didn’t have a “-”) must be the earn options. There was also some slight confusion over what the image above the -8,000 represented.

Outcome: We adjusted the image so that it looked closer to a receipt. We also added a plus sign “+” next to the earn options so that users can immediately spot those items that earned Kin.

Note: The details presented on these screens were intended for research purposes only and are not final.

Last Thoughts: Product development is an iterative process. Usability testing allows us to see how potential users would interact with our product as we’re building it, which can help us make products that provide a user experience that is clear and easy to understand for any potential user. We’ve learned a lot from our users’ insights so far, and look forward to sharing more research with you as we go.