Demographics are a spectrum
Stop surveying with radio buttons and ‘other’ boxes
As part of my work for the nonprofit Uplift, I regularly put together surveys for convention attendees through our Elephant at the Con initiative. An important part of the survey is demographic information. We want to know how harassment is correlated to certain demographics. We made a decision early on to use free-form text instead of radio buttons when asking about gender, race, and sexuality. It takes a bit of extra work to format the data correctly, but the benefits of richer data and inclusion far outweigh the extra hour of formatting.
We have conducted these surveys at many different conventions for over a year, so I am pretty familiar with the responses we get, but this September’s survey at LeakyCon featured a largely European audiences and suddenly the responses I received for race that I received were: British, Irish, White — European, etc.
In this case I could have predicted this by noting the location of the conference and researching race categories used in European surveys, but realistically you are never going to fully research your audience before conducting the survey. In fact you are conducting the survey to research the audience and by forcing people to bucket themselves into categories that do not completely describe them, you skew the data and miss out on the opportunity to see some trends.
The above example indicate show even white people can benefit from free-form text surveys, but more commonly multiracial people are forced to choose just one race to identify with, asexual folks are left out, and nonbinary identities are forgotten.
Free-form options for demographics make survey respondents happier. People who always have their identity listed at the top may be annoyed that they have to type out a response, but many others will appreciate the option to define themselves using their preferred terms. Free-form text fields allow people to be as specific as they would like to be. People may want to leave these questions blank or say they are not sure.
Think of a facet of your identity that is marginalized. Wouldn’t you appreciate the opportunity to define yourself in your own words? If you can’t think of an example, imagine what it would be like if your gender or race was not listed on a survey form.
Case Study: SurveyMonkey Sample Demographic Template
These survey templates are used by thousands of surveys every year and though they are inspired by the US Census data survey, they should be improved. SurveyMonkey templates set a precedent for survey makers.
Pros: Free-form text fields, acknowledges multiracial identities.
Cons: Free-form text fields are designated as “other” and only used for groups they left out. Buckets multiracial people into a monolith instead of allowing them to check multiple races.
Case Study: Google Docs
Google Docs do not have sample demographic templates, so the results are varied. Here are some examples.
Case Study: Qualtrics Surveys
Commonly used for marketing and user research surveys, these surveys tend to be high-tech, but not very inclusive.
Pros: Includes a field for mixed race and allows you to specify if other races apply to you.
Cons: Implies that all black people are African-American, uses Latino/a instead of Latinx, while conflating Latinx identity with race (more details). Doesn’t allow you to check multiple boxes as there is just a mixed race option, and it includes a free-form text field but it is marked as other.
Case Study: Lesbians Who Tech
Lesbians Who Tech is a group for Queer Women (and Allies) in the tech space. I attend a lot of events with LWT and love the group, but despite having a tagline of “queer, inclusive, and badass,” the surveys they use for their events are quite limiting.
Pros: A variety of identities, check boxes. Uses Latinx. Acknowledges intersecting identities.
Cons: Identities like bisexuality, asexuality, pansexuality, intersex, agender, are all left out and the only way to enter this information is to be bucketed into LGBTQ, Queer, or Other. They are missing a lot of valuable data as a result. A lot of races are left out, presumably because they are not part of the three most underrepresented races in tech.
Case Study: Granger Leadership Academy
GLA is an activist / leadership conference put on by the Harry Potter Alliance. Their demographic surveys are always great. Allowing people to include either a sexual or romantic orientation (or both) is a nice touch.
- When making survey websites, include an option for free-form text that isn’t labeled as other.
- Allow free-form text as the default unless you absolutely need split categories and in that case: (1) do you research and include a lot of categories, (2) use checkboxes not radio buttons, and (3) no other boxes.
- Ensure that diverse viewpoints are contributing to the development of your survey; there is always room for improvement!
- See a survey that takes a binary approach to gender or assumes people are only one race? Private message the surveyor and ask them to improve. Change starts with you!