A first version of adify.be was released on April 15th, a little more than two weeks after we published our initial planning.

Goal

The main goal of the first iteration was validating the usefulness and usability of a location-based advertising platform.

Additionally, we wanted to get answers to the following questions:

• Does a random visitor understand the purpose of the website?
• Are users able to execute basic operations such as browsing, starring and creating advertisements?
• What functionality is not being used or is missing?

Method

We choose to make use of the think aloud protocol. We evaluated using the following pattern:

• The user was asked to navigate to http://adify.be without further instructions. The user was let to explore the website while thinking aloud.
• If not done spontaneously, we asked a user to create an advertisement, to view an advertisement and to mark an advertisement for future viewing.
• We asked them to show the advertisements they had just created and favorited.

From here on out, the user was asked some specific questions, such as why they did or did not perform certain actions. Finally, we asked them what they thought was missing or needed improvement.

Rationale

In the early stage of development, we wanted extensive feedback from our test subject by interacting with them during the test. This is not possible with a questionnaire without influencing the final results. Additionally, a SUS questionnaire limits the scope of testing to a list of predefined questions, while think aloud on the other hand allows the user to provide insights on things we, as developers, might not have in mind.

A disadvantage worth mentioning of the think aloud approach is that it will be hard to compare results next iteration. We took this into consideration, but felt that the advantages of getting key problems out of the way early on outweighed the need to quantify the improvements.

Results

• Although creating an advertisement was found to be straightforward, users found it unclear what the buttons in front of some of the input fields were for. None of our test subjects made use of them, nor where they aware of their purpose.
• The image placeholder, shown as a rectangle with a ‘+’ sign in the middle, was unclear to our users. Replacing them with an icon of a photograph was sufficient to fix this issue.
• Starred advertisements are called “Watched Ads” which caused users to think it contained a listing of advertisements opened in the map view. Renaming this section to “Starred Ads” solved the issue.
• Closing advertisements was way to hard, the close button was small and users expected that clicking outside the popup would close it.
• When starring an advertisement, the user received no feedback. It was unclear whether the classified was effectively starred or not.

Conclusion

Thanks to the feedback of our users we spotted some essential missing features that dramatically influence the website’s usability.