The Data on Dropdowns

Beer van der Drift
Nov 9, 2020 · 4 min read

This post was originally posted on my blog here

Scroll scroll scroll. Going through forms and having to open a dropdown to select your country, or even worse, gender can be time consuming. I’ve been wondering how time consuming exactly and whether some select types are better. So I thought; let’s run an experiment!

I quickly hacked together some code and let over 500 unique users fill in a form asking for their age and country. The app asks users to select their country from 3 to 55 options (Distributed according to the Fibonacci sequence, because I like Fibonacci 🤓 ). The app shows either of the following four types of select interactions:

Image for post
Image for post

The results were as follows (click on the image to open the actual chart):

Image for post
Image for post

Why does the drop-down spike at 21 options?

Since all the respondents were from the States (I had to guarantee one option somehow) and the “U” is low in the alphabet, my assumption is that somewhere above 13 options is when the option started to fall off the screen.

Why is autocomplete so high?

That’s also what baffled me, I expected this to be the fastest way to do a select when there are many options. There are two hypotheses:

In a scatter plot you can clearly identify that a good amount of people who had to use the autocomplete interaction struggled to select their answer.

Image for post
Image for post

The “Autocomplete” outliers were at least three times as frequent as the others.

Since the autocomplete outliers were overrepresented, I assumed that the autocomplete were caused by something else than maleficent intent and decided not to remove the autocomplete outliers in the data-cleaning process (explained further down).

💡 Idea: re-run the experiment with “The” in front and a nudge to let users know they have to click the option. But, since I used Mechanical Turk to source the responses and this is just a fun experiment, it wasn’t worth the 💰!

What are some biases?

A quick experiment doesn’t come without biases. First off, the ages of 20–49 are over-represented;

Image for post
Image for post

💭 Thought: the data of this experiment is representative for you if your demographic matches these responses. If your user’s age distribution is very different you might want to consider re-running the experiment yourself for your demographic! Feel free to use the code here!

Also, over 80% of respondents is from the US 🇺🇸, with the rest being Dutch 🇳🇱 (I trialed it first with friends!).

Image for post
Image for post

💭 Thought: since users had to select their country, the positioning of “United States” and “Netherlands” in a list might have introduced a bias in the data.

Data Cleaning

Of course there were also some jokers 🤡! I removed any response where:

Conclusion

With an average of 18 data-points for each combination there is too little data to make strong recommendations, but I’m going to try anyway:

Want to check out the results yourself? Check out all visualization here! Want to run the experiment yourself? Check out the code here! Looking for the raw JSON pre-cleaning to do your own analysis? Go here!

The Startup

Medium's largest active publication, followed by +754K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store