CUTGroup Miami #11: CivicPro, Phase 2
Understanding what it means to be “civically engaged.”
As part of the Civic 101 Lecture Series hosted by Code for Miami, CUTGroup Miami members were invited to attend a workshop that included three stations related to accessing and utilizing government information. In addition to a hyper-local civic engagement presentation by Matt Haber of CivicPro, participants rotated through an Agenda Feedback Form, a Civic Knowledge Awareness Test, and lastly, a Card Sorting exercise.
This session took place July 31, 2017 at the CIC Miami.
What were the goals for the session?
Part 1: Understanding agenda feedback form comprehension
The primary goal of this section was to better understand how citizens with varying levels of civic experience comprehended sample agenda items provided by CivicPro, as well as to observe participants comfort level with the lexicon of government agenda items. Recipients were given an agenda item, then worked with a proctor to answer the following questions:
- Which keywords did you find most important in the agenda item?
- What was this legislation about?
- Which policy issue category or categories did it fall under?
- If the agenda item were passed, please explain what it would do?
- Did you find this item interesting?
Part 2: Evaluating general knowledge of existing civic information through an awareness test
The civic knowledge awareness test was developed by CivicPro and can be seen available here: http://bit.ly/CivicsTestMiami. It’s important to note that while this was in the form of a quiz with right and wrong answers, the goal was to get a general idea of basic civic knowledge from a sample size of people living in Miami-Dade County.
Part 3: Categorizing themes through card sorting
We used card sorting as a way for participants to categorize cards under similar themes to better define title headings (category names).We also wanted to create new, previously unrecorded category titles that would better encapsulate themes than the existing card titles.
Responses
A total of 12 completed proctor guide response forms with signed releases were submitted and accepted. Eight participants were already members of the CUTGroup email list, while four attendees had no CUTGroup affiliation. While two participants participated in previous CUTGroup testing events, ten attendees were new testers.
Part 1: Agenda Feedback Form
“The verbiage is a little ambiguous.” — Participant
- We received twelve submissions.
- There were some technical issues with this particular section: Agenda Item #6 had no responses, since none were assigned. Also, Agenda Item #8 not included in drop down.
- Item #10, both participants used “sharing economy” as most important keywords.
- 10 out of 12 participants found their agenda item interesting. The other two participants marked agenda items “not interesting” for items related to buses & surveillance camera procurement.
Part 2: The Civic Knowledge Awareness Test
- We received fourteen submissions total.
- The average score of a submission was between 50% to 60%.
- “Most” knew who the mayor was.
- 10 out of 15 recipients didn’t know The Children’s Trust in Miami was a Government Entity; in fact, it’s a “dedicated source of revenue established by voter referendum to improve the lives of children and families in Miami-Dade County,” as noted on their homepage.
- Error on last question (always -1).
Part 3A: Card Sorting Exercises
- “Prof. Torres” (Aerial): County Managed, Fed Managed, Local Industry (private), Locally Run (people) and Private Industry
- “HER” (Aerial): Public Works, Economic Development, Government Budgets & Finance, Hospitality, Community Events, Economic Growth, Civil Rights
- “Megan” (Aerial): Health, Local Gov (less priority), Civil Rights, Local Gov (High priority), Federal, Tourism, Freedom of Speech, Other.
- “Camille” (Aerial): Zoning & Land Use, Homeland Security, Public Health & Safety, Community Development, City Tourism, Parks & Recreation, Infastructure Development, & Civil Services
- “MiamiMartin” (Aerial): (in action) Other, Crime & Safety, Public Land, Government Budgets & Finance, Environment, Community Service, Economic Development, Healthcare, Civil Liberties, Infrastructure
- “Susana” (Aerial): Sea-level rise. Sustainable jobs. Equitable development. Transit-oriented development. Voting issues. Public space as amenities.
- “John” (Aerial): Ideal Spending, Gentrification, Environment, Gov. Transparency, Economic Development, Rest of Country, Internet Access, South Beach Weekend
- “Cayla” (Aerial): Government Transparency, Public Records, Civil Rights, Economic Growth, Health, Homeland Security, Children & Youth Programs, Land Use, Environment (Did not finish)
- “Sam” (Aerial): (in action) Government Transparency, Environment, Fun, Economy, Public Health, Law Enforcement, Infrastructure, Land Use
- “Camilo” (Aerial): Education, Economic Development, Environment & Climate Change, Crime & Safety, Constituent Groups, Infastructure, Public Services, Technology & New Industries, Government Budgets & Finance, Land & Housing, Heathcare
- “wonderwomanjc” (Aerial): No Category, Education, Health, Crime & Safety, Government Budgets & Finance, Economic Growth, Environment, Government Transparency, Public Works
3B. Card Sorting Analysis: Most Common Category Used
- Government Budgets & Finance, 4x
- Economic Development, 4x
- Crime & Safety, 3x
- Government Transparency, 3x
Notable differences: Federal vs Local and Sea Level Rise/Climate Change were not included in our original card titles but were suggested to be included on more than one occasion.
Testing session challenges
- An intense rainstorm descended into the Miami area on the Monday night we were scheduled to host, which drastically reduced expected turnout.
- While we anticipated many participants attending the Civics 101 Lecture series to spill over into our testing session, only a handful did.
- The planned time allocated for card sorting deviated massively from person to person and thus, created bottlenecks in our process. At one point we, unfortunately, had to tell a participant to make final decisions and move on, as there were “too many cards and not enough time.”
- We also noted specific instances in the card sorting exercise where participants were overwhelmed with the sheer number of categories to review, and immediately showed signs of decision fatigue. In one case, categories were deemed “unnecessary” and left out entirely.
Conclusion
This research and development phase of the CivicPro platform yielded valuable insights into community information usage and comprehension. While we observed participants reviewing the provided agenda item materials, many explained that they were not entirely sure what the articles “actually” meant. Some could recall the same keywords, and ten of our twelve participants thought the agenda items, on the whole, were “interesting.” The interest from test subjects shows, despite much-needed improvements, that folks in the community are willing and able to engage with Government information.
We’ve learned a citizen’s civic knowledge can vary widely and almost always dependent on their current level of local civic participation. For example, while most of our participants knew who the mayor of Miami-Dade County was, many didn’t know The Children’s Trust was a government entity approved by voters for the public’s benefit.
The card sorting exercise demonstrated how difficult it is to separate local legislation into a manageable number of discrete categories, due to the variety of policies handled by local government and the way many of those policies interrelate with each other.
No magic bullet will solve the problem of civic engagement, so we must consider these experiences when rethinking how participants would benefit from the presentation of civic information.
