How we used a simple survey tool to elevate public engagement

By Edward Paulino, Fellow, Office of Innovation, State of New Jersey

“Please take our survey.” Four words practically guaranteed to get your message dumped into spam folders or disregarded. Yet, when government seeks much needed public input, surveys are often one of the first tools to be deployed.

One tactic to increase response rates is to keep surveys short. Generally, the shorter the survey, the more likely someone is to complete it. However, streamlining surveys inherently sacrifices the amount of data collected and the depth of insights that can be gathered.

With the help of a tool called All Our Ideas, however, survey designers can present a simple choice between two options, but still obtain valuable learnings. All Our Ideas uses a powerful algorithm that digests individual responses and produces insights that allow for the comparison of hundreds of choices.

The New Jersey Office of Innovation has deployed All Our Ideas to support a number of projects. Examples include campaigns that seek public input on K-12 education priorities, climate change solutions, and the upcoming challenges facing workers. In total, over 16,000 people responded more than 768,000 times across the engagements.

As a result, the public helped the State of New Jersey develop climate change strategies, informed the creation of a policy roadmap to improve the health, safety, and economic security of workers, and identified the top concerns of teachers, students, and caregivers.

This level of participation is made possible by the platform’s simplicity. Those taking the survey are presented with two choices to pick between. Once a user selects which statement they prefer, a new pair of choices from the larger set of options is revealed. This process repeats itself, allowing participants to respond as many times as they’d like. Additionally, because the order of item pairings is random, it is resistant to organized attempts to influence the outcomes.

While the user is only presented with a couple of options, behind the scenes the platform coordinates comparisons between hundreds of options. As more people participate, the ranking of the options becomes more accurate. The outcome of the individual responses determines a given option’s score. This score represents the probability a participant will select that option over a randomly selected option.

Participants can also submit their own options. When a new option is submitted, it is reviewed by the creator of the survey to monitor for appropriateness, and, if it is accepted, the idea will start appearing as an option under the corresponding question.

This survey method has a number of advantages over traditional approaches. First, it is simple to use — to mark your preference, just click on that option. By making the survey easy, it allows more people, especially those with limited time, to participate. At no point is a user bombarded with an overwhelming number of choices that discourages engagement.

Second, users can respond to as many questions as they would like, submit as many ideas as they would like, or stop at any time. This allows the participant to right-size their engagement. This “high ceiling/low floor” level of engagement opens the door to more people and makes the most of those who would like to contribute more than average.

Despite its advantages, All Our Ideas is not a one-size-fits-all solution. For example, it is not designed to gather qualitative data.

Additionally, because the tool does not collect information about the individual participating, you cannot tie preferences to different groups within the audience. Wanting to tease out, for example, the respective preferences of educators, students, and caregivers, you would have to use three different instances of the tool. The same limitation exists for organizing the data into categories — it would require an instance for each category.

Despite not checking every box, All Our Idea’s simplicity for the end user and breadth of scope for the survey facilitator make for a powerful combination. In the ever escalating battle for user attention, sometimes less is more.

Interested in using the All Our Ideas platform for an upcoming engagement? We’ll be sharing a detailed guide for using the tool in an upcoming post. This post will cover optimal use cases, a step-by-step deployment tutorial, and best practices for launching a successful engagement. Make sure to subscribe to our newsletter to get the latest updates.