CUTGroup Miami #12: CivicPro, Phase 3
For the third and final phase of the CivicPro and CUTGroup Miami collaboration, three versions of an email alert registration feature was tested by Miami residents for usability, display preference, and understanding. Key elements of CivicPro’s final product would be informed based on each participant’s experience and feedback. While these wireframes only showed a basic representation, it allowed for conversations around expectations to occur, an invaluable aspect in the design process.
This session took place November 2, 2017 at Venture Cafe Miami.
About the wireframes:
Matt Haber, Co-founder of CivicPro, used InVision, a popular industry prototyping tool to create the following samples used for the purposes of this testing session.
Version #1 (left)
The first version prompted users to sign-in then displayed all topics (as defined by the phase 1 survey) an interested resident may want to subscribe to receive in the future. After selecting any of the displayed topics, the participant was prompted to “select a local government” and once chosen, an alert set confirmation would appear. During this time, proctors took notes in response to participants reflections on the overall journey, likes, and dislikes. As a lead-up to Version #2, a question was posed related to the categorization of park information, allowing the selection of up to three topics they thought was a likely associated.
Version #2 (middle)
The second version prompted users to sign-in then displayed a search box (as pictured). Participants then using a sample query, “Parks,” would display three topic categories associated with parks; i.e. Environment, Government Services & Infrastructure, and Youth and Senior Programming. After selecting, the “select a local government” screen was displayed and lastly, the alert confirmation would follow government selection. The design elements tested in comparison to Version #1 includes the initial search query screen and vertically aligned search results.
Version #3 (right)
The third version prompted users to sign-in, then like Version #2, displayed a search box that participants could query using the sample “Parks.” A horizontally aligned results would appear below the search box, and after selecting, would lead to the local government screen then finally, once a government was chosen an alert would be set. The design elements tested in comparison to Version #2 includes extra copy, “I want to be alerted when” above and “is discussed” below the search box, as well as horizontally aligned search results.
About the participants:
CUTGroup Miami members were invited along with the general public in attendance at Venture Cafe’s Thursday gathering to our testing session. Eight folks we met with noted they were not already registered as Civic User Testers while nine were pre-scheduled. Seventeen total participants completed seventeen design feedback surveys. The multiple choice portions of the design feedback survey can be found here, with select qualitative portions quoted below. Six attendees selected the 25–34 age group, three selected 35–44, three selected 65 and above, two selected 45–54, two more selected 55–64, and one person preferred not to answer. Six identified their race/ethnicity as Hispanic/Latino, five as White/Caucasian, four as Black or African American and one person preferred not to answer. Employment ranged from housekeepers to students, to a non-profit program director and banker. Twelve of the seventeen Miami residents tested for the first time!
Results
Version #1
Overwhelmingly, participants preferred this version with just four choosing Version #2 over #1. “Easy to use” and “clear” was referenced multiple times, as a shared consensus in its user-friendliness and simplicity can be understood. The icons were liked by many participants while some described a need for better forms of confirmation in the setting of alerts. As one participant noted when setting a new alert it, “looked too much like previous page.” When asked if participants, based on what was seen so far, thought they had an idea of the tools “does” most were correct in their understanding an alert would be sent via email later however some expected deeper resources, such as for taking action or to “inform you of city protocols.” While questions of a potential lack of information and email arrival schedules remained, Version #1 had overall supportive evaluations.
Version #2
Participants preferred this version second most, describing the search feature as “very clean” and “simplistic.” Other positive comments included the use of a search query “kind of educates and nudges” to the associations between services and those terms use. In two cases of concern, a participant described that they needed the displayed topics “as a guide” and another noted it would be “harder to search” if you didn’t know the topic, an important consideration where icons could help. About eleven folks felt the copy used to describe the search page failed to adequately explain it.
Version #3
While very similar to Version #2, none chose this version as most preferred. Eight of seventeen surveyed said they did notice the sample “Parks” search results changing from vertical (in Version #2) to horizontal (in Version #3), while four did not notice but still preferred horizontally displayed results. Just four participants noted the extra copy added above and below the search box did not adequately explain the page, while thirteen now preferred the extra language. As noted by one participant, they “didn’t quite understand in the second version that I would be alerted every times this topic comes up.”
Reasons for choosing:
Version #1:
“More information without having to search.”
“Clear and covers more topics.”
“More inviting. More icons led it to look more put together, better developed.”
“[Version #2 & 3] doesn’t give me more information until I’ve been presented with a (guided, gentle) challenge.”
“More choices = more options for alerts.”
“It had all options.”
Version #2:
“Search with no extra text.”
“[Version #1] feels like a lot of clicking to get what you want.”
“Fastest, least amount of reading or confusing text.”
“Version 3 is confusing.”
Version #3
-No Comments-
Testing Session Challenges
Hosting a design feedback session during Venture Café provided a great venue and central meeting point for both testing participants and test proctors, however due to the size of the room and the conversations taking place outside, it was slightly difficult to hear each other speak. There was also just two and a half hours available to conduct our experiments in this phase as compared to phase two’s three hour window, which limited our bandwidth to accommodate potential walk-ins.
Conclusion
This final research phase of the CivicPro tool provided several insights that would have been missed without this crucial qualitative step. Simplicity and ease-of-use should remain the most important value in continuing iterations, above any other considerations. The language used to describe the setting of alerts should flow more intuitively towards the scheduler’s goals, as a recommendation to “check your email for a just sent sample now” could provide.
While Version #1 may have proved the most popular design mock-up, with icons being strongly recommended, more than one participant commented on the potential for including a search feature at the top of the Version #1 home page. In that way, one could design the interface in a way that satisfies both those who want to see a full list and those looking to filter immediately without reading more than needed. In review of the displayed “parks” search data populating horizontally versus vertically, the results did not show a large enough response to warrant one being preferred significantly more over the other. Another improvement request by one participant noted the need for the previously set alerts to show an effect of change; as in a green check mark or shadow showing it’s been pressed or subscribed too already.
With participants ranging age ranges and careers, this feedback represents real world examples of the roadblocks and challenges locals will face when using digital services. Now, with those experiences and conversations at the center of the design process, CivicPro can be in the best position to become truly successful in achieving it’s mission; to provide the tools to effectively participate in the policy making process.
