Test #1: The City of Miami’s Alpha Website
A first pass
For our first Civic User Testing Group (CUTGroup) session, we tested the City of Miami’s Alpha website, an updated, redesigned and reorganized take on their current website. While the website provides valuable information for both City of Miami residents and visitors, its primary purpose is to enable Miami dwellers and business owners to find the how, where, and what of anything related to city-specific tasks and items. It seeks to provide efficient and practical information and tools for those looking to accomplish specific city-associated tasks, conduct business within the city, or collect information about the area.
On their current website, users can apply for all sorts of permits, find out about upcoming events, time their commute with the Trolley Tracker, read the latest news from city officials, and much more. The new version of the website will provide the same information but in a more visually pleasing and intuitive format, making it more efficient to interact with and navigate.
The City of Miami partnered with CUTGroup Miami because they wanted to observe and understand how regular City of Miami folks would respond to the alpha website, especially before the website launches. City of Miami developers and staff were eager to learn how to improve the site further to provide the most seamless and efficient experience for the user.
Segmenting
On November 7, 2016, we sent an email to 142 CUTGroup testers whose zip codes matched those that fall within the City of Miami limits. We wanted to know if they would be available for an in-person test on November 15, 2016. When segmenting our testers, we were interested in City of Miami residents, as the alpha website provides information specific to only the City of Miami. We wanted to include testers who had and had not used the current City of Miami website. We were also interested in hearing from testers who were and were not small business owners. Lastly, we wanted to include testers with different types of devices.
Screening Questions
In response to our initial email to 142 City of Miami residents, we heard back from 30 testers who indicated they were interested in participating. We gathered interesting information just from the screening questions.
87% had used the current City of Miami website before, for the following purposes:
- Events and News: 47%
- Parks and Recreation: 47%
- Maps: 40%
- City Directory: 30%
- Other (complaints, passport, recycling, employment, grant writing): 20%
- Building Permits: 17%
- Emergency Services: 10%
- Financial Transaction Search: 7%
- Neighborhood Enhancement Team (NET) Services: 3%
Two of the 30 interested testers were small business owners.
Test Format
For this in-person test, each tester paired with a proctor, either a City of Miami employee associated with the alpha website project or with CUTGroup Miami. The proctors prompted the testers to complete various tasks and recorded their interactions as they did so, including their verbal and nonverbal responses, reactions and feedback.
All testers were asked to review the alpha website homepage. Following this, testers were directed to the current City of Miami website and prompted to complete different tasks.
Testers used their laptops, tablets, or smart phones to complete the tests.
Results
On November 15, 2016, we tested the City of Miami Alpha Website with four CUTGroup testers at the City of Miami staff offices, located in the Downtown Miami neighborhood.
Homepage
Reaction to the alpha website homepage was varied. Some testers appreciated the colorful background image, while others said that it was “too artsy” and not in keeping with how a government page should look.
Several testers questioned how the services displayed in the center of the page had been selected, and said that the most popular, searched-for services should be prominently featured. Testers appreciated the front-and-center placement and size of the Search Bar, as it was immediately evident to any page visitor.
One tester expressed confusion regarding the Navigation Bar’s usability:
“How does the Nav Bar fit in with the services? Am I supposed to look in the Nav Bar to find something or do I look in the Services section?”
Most of the testers felt similarly about the purpose of the website, explaining that it provided information about services specific to the City of Miami; however, one tester expressed uncertainty about what the website does.
When prompted for their opinion on the website’s intended audience, testers agreed that it was designed primarily for residents of the City of Miami. They decided that it was a useful website for visitors too. Some testers believed the site’s design for a younger audience in mind because of the images and styling.
Ease of Use & Language
Testers were questioned about the usability of the current City of Miami website, as many of the assigned tasks had not yet been incorporated into the alpha version.
Though testers were assigned different tasks, most rated their duties to be difficult or very difficult to complete. All testers tasked with applying for a permit were unable to immediately find where to go/click to access the application. All clicked on two or more wrong paths before finding the correct way to the permit application.
Testers tasked with applying for a film permit or a permit to build a fence recommended a simpler, more user-friendly process, including displaying a list of steps required before beginning the actual application, as well as incorporating more straightforward language that the average person would understand.
“[This permit application is] not good for a resident who’s trying to understand what to do. [They are] obviously expecting a user who knows what they’re doing. It seems that they really want you to come in person.”
Testers did not appreciate the amount of reading required on the permit application forms, saying it was “too much.”
One tester tasked with applying for a Garage Sale permit said the Search Bar “worked well as an alternative method” to finding the correct page. The tester said the application pdf form itself was not suitable for a mobile device user.
Next Steps
Once the CUTGroup test was completed, the results were collected and summarized in a Final Report which was shared, along with the raw data, with the City of Miami team. They intend to peruse and apply the feedback to further development and improvement of their alpha website, which they hope to launch soon.
As for CUTGroup, this was a small but very important test, as it not only provided a stellar learning experience but also set the tone for user tests to come.
