Demographics are a spectrum

Stop surveying with radio buttons and ‘other’ boxes

As part of my work for the nonprofit Uplift, I regularly put together surveys for convention attendees through our Elephant at the Con initiative. An important part of the survey is demographic information. We want to know how harassment is correlated to certain demographics. We made a decision early on to use free-form text instead of radio buttons when asking about gender, race, and sexuality. It takes a bit of extra work to format the data correctly, but the benefits of richer data and inclusion far outweigh the extra hour of formatting.

Example demographic portion of our survey: what is your gender, what is your race, what is your sexuality?

Richer Data

We have conducted these surveys at many different conventions for over a year, so I am pretty familiar with the responses we get, but this September’s survey at LeakyCon featured a largely European audiences and suddenly the responses I received for race that I received were: British, Irish, White — European, etc.

In this case I could have predicted this by noting the location of the conference and researching race categories used in European surveys, but realistically you are never going to fully research your audience before conducting the survey. In fact you are conducting the survey to research the audience and by forcing people to bucket themselves into categories that do not completely describe them, you skew the data and miss out on the opportunity to see some trends.

The above example indicate show even white people can benefit from free-form text surveys, but more commonly multiracial people are forced to choose just one race to identify with, asexual folks are left out, and nonbinary identities are forgotten.

More Inclusive

Free-form options for demographics make survey respondents happier. People who always have their identity listed at the top may be annoyed that they have to type out a response, but many others will appreciate the option to define themselves using their preferred terms. Free-form text fields allow people to be as specific as they would like to be. People may want to leave these questions blank or say they are not sure.

Think of a facet of your identity that is marginalized. Wouldn’t you appreciate the opportunity to define yourself in your own words? If you can’t think of an example, imagine what it would be like if your gender or race was not listed on a survey form.

Case Study: SurveyMonkey Sample Demographic Template

These survey templates are used by thousands of surveys every year and though they are inspired by the US Census data survey, they should be improved. SurveyMonkey templates set a precedent for survey makers.

Pros: Free-form text fields, acknowledges multiracial identities.

Cons: Free-form text fields are designated as “other” and only used for groups they left out. Buckets multiracial people into a monolith instead of allowing them to check multiple races.

Case Study: Google Docs

Google Docs do not have sample demographic templates, so the results are varied. Here are some examples.

This Stanford Crowd Course survey leaves out a lot of people.

This is better, but still leaves out folks.

Prefer not to say is a good addition, but it does force anyone else to select “other”

A bit better with a free-form text field, but the fact that it’s marked as other is negative.

Much better! By allowing people to define their identities, they get better results. This survey had a similar race question.

Case Study: Qualtrics Surveys

Commonly used for marketing and user research surveys, these surveys tend to be high-tech, but not very inclusive.

Pros: Includes a field for mixed race and allows you to specify if other races apply to you.

Cons: Implies that all black people are African-American, uses Latino/a instead of Latinx, while conflating Latinx identity with race (more details). Doesn’t allow you to check multiple boxes as there is just a mixed race option, and it includes a free-form text field but it is marked as other.

Case Study: Lesbians Who Tech

Lesbians Who Tech is a group for Queer Women (and Allies) in the tech space. I attend a lot of events with LWT and love the group, but despite having a tagline of “queer, inclusive, and badass,” the surveys they use for their events are quite limiting.

Pros: A variety of identities, check boxes. Uses Latinx. Acknowledges intersecting identities.

Cons: Identities like bisexuality, asexuality, pansexuality, intersex, agender, are all left out and the only way to enter this information is to be bucketed into LGBTQ, Queer, or Other. They are missing a lot of valuable data as a result. A lot of races are left out, presumably because they are not part of the three most underrepresented races in tech.

Case Study: Granger Leadership Academy

GLA is an activist / leadership conference put on by the Harry Potter Alliance. Their demographic surveys are always great. Allowing people to include either a sexual or romantic orientation (or both) is a nice touch.

Moving forward

• When making survey websites, include an option for free-form text that isn’t labeled as other.
• Allow free-form text as the default unless you absolutely need split categories and in that case: (1) do you research and include a lot of categories, (2) use checkboxes not radio buttons, and (3) no other boxes.
• Ensure that diverse viewpoints are contributing to the development of your survey; there is always room for improvement!
• See a survey that takes a binary approach to gender or assumes people are only one race? Private message the surveyor and ask them to improve. Change starts with you!

Did you learn something?👏 to spread the word. To learn more about Uplift and our Elephant At the Con survey, visit our website. Have thoughts? Let’s discuss in the comments or on twitter.