The Potential of Voice-Input Surveys

Written by Maya Crowden, Zihan Zheng, Tam Fleming, Anusha Bayya

Surveying during COVID

As businesses have struggled to adapt to evolving COVID-19 restrictions and reopening strategies, many have turned to new technologies to try to understand how consumer sentiment is changing. Most popular survey tools, such as Google Forms and SurveyMonkey, rely on users to select multiple-choice answers or type their responses. At Basil Labs, we designed Ella, a voice-input survey software, in hopes that this format would encourage consumers to provide longer, more thorough answers to open-ended questions. By using a tool that would elicit more detailed answers from customers, businesses are enabled to develop a clearer understanding of consumer sentiment and better target areas for future improvement.

Photo by Ekaterina Bolovtsova from Pexels

What maximizes response quality and length?

We wanted to see how the more common form of surveys, asking consumers to type their responses, compared to Ella when consumers were presented with open-ended questions. Would there be significant differences in user responses to open-ended questions? Is one type of survey better for helping businesses understand consumer sentiment?

We sent out identical survey questions using Google Forms and Ella. We chose to use Google Forms to represent the more traditional survey format due to its accessibility and widespread use, so participant responses on Google Forms would not likely be affected by user error that could lead to misleading results.

After examining the overall length of responses as well as the frequency of different parts of speech, we found there was a significant difference in the responses based on which survey tool was used. On average, Ella surveys received responses that were over twice as long as the responses to the Google Form surveys. Breaking the responses down into parts of speech, Ella surveys typically received double the number of nouns the Google Form surveys received. Ella surveys also received responses containing almost triple the number of adverbs and adjectives, suggesting users are not only talking about a greater number of topics but are also willing to provide more detailed information.

Ella by Basil Labs

By the numbers — a closer look at our study

The mean length of responses, including spaces, to each question, was 184.1 characters for the Ella survey and 80.0 characters for the Google Form survey. Both surveys had the same standard deviation of 64.4. We also examined the impact of including the stop words in our analysis of participant responses. Regardless of the inclusion of stop words, Ella consistently produced longer responses. Including stop words, the average length of responses per question was 42.5 words for Ella compared to 14.9 words for Google Forms. Removing stop words, the average length of responses per question was 23.8 words for Ella and 10.5 words for Google Forms.

We also broke down the responses to each question by parts of speech to examine which survey tool led to more detailed responses. Each variable we examined had a higher mean value associated with the Ella survey compared to the Google Form survey.

  • Nouns: 6.4 (SD = 3.7) for Ella; 3.2 (SD = 2.0) for Google Forms
  • Verbs: 4.1 (SD = 2.2) for Ella; 1.4 (SD = 1.5) for Google Forms
  • Adverbs: 4.0 (SD = 2.4) for Ella; 1.1 (SD = 1.2) for Google Forms
  • Adjectives: 3.1 (SD =1.7) for Ella; 1.4 (SD = 1.1) for Google Forms
  • Clauses: 2.7 (SD =1.0) for Ella; 2.4 (SD = 1.2) for Google Forms

A future in open-response, voice-input surveys

All of these data points suggest the Ella survey may lead participants to maintain a higher level of interest throughout the survey, enabling businesses to obtain a larger quantity of higher quality information compared to more popular survey tools. The difference in length and level of detail is significant for businesses and researchers who are trying to understand how COVID-19 and the vaccines are impacting consumer preferences and want to maintain an up-to-date understanding of consumer sentiment.




Understanding Ella’s ability to interpret and visualize open-ended survey data, serving as a powerful tool in the pursuit of creating the ‘ideal survey,’ geared towards collecting qualitative data through surveys that encourage open-ended thought and use of voice

Recommended from Medium

Enterprise Data Warehouse Design and Implementation

“ Unexpected Question“

You Can’t Boil the Ocean: How Data Governance Teams Bring Focus

Very Basic Twitter Stock Sentiment Analysis

Data Science Internship as LetsGrowMore community

[Latest] SNPV Time table Admit Card download | Shaheed Nandkumar Patel vishwavidyalay Time Table…

SQL Cheatsheet- OVER

3 Python Modules You Should Know to Extract Text Data

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Basil Labs

Basil Labs

Exploring the applications of AI in consumer analytics.

More from Medium

One Way Other’s Wisdom Can Set The Tone Of Your Speech

Making cognitive labor visible and tangible — WIP

GO briefing // Q1 2022

Plant Life, Form, and Protection