How Culture Influences Debugging
When I moved to Peru a decade ago, I began to notice how much of the computer programming world was centered in the US (and within the US, Silicon Valley). I began to wonder who around the world had the most opportunities to learn programming, and how people around the world might program differently.
After starting graduate school I started to learn about how researchers have found that culture influences how people use computers in general. One of those differences is that some people prefer more step-by-step instructions, whereas others don’t.¹ So I and my co-authors wondered if we could also see differences in how people use a programming tool: Python Tutor.
Python Tutor is an online debugger with a particularly interesting feature. It executes code you enter and then lets you move forward and backward through the execution while displaying the variables. We call any movement to an earlier step in an execution a “back-step.”
We analyzed six months of data from Python Tutor (1 million code executions from 69 countries) to see if back-step use varies between countries and cultures. When we did this we found back-step use indeed varied between countries, and in particular, it varied with the self-directed learning of those countries.
In societies with self-directed learning, students are often encouraged to find their own ways of solving problems, and these students were more likely to use back-steps. In societies with instructor-directed learning, students often expect their teacher to tell them exactly how to solve the problem, and these students were more likely to go forward through the code execution without taking back-steps. In summary: Instructor-directed learners take fewer back-steps.
Next we created two debugging tasks and had people around the world work on those tasks and answer some questions about their personal values related to instructor vs. self-directed learning. We had 522 people from around the world try our debugging tasks.
We found (contrary to our expectations) that people who used more back-steps generally did worse in debugging. However, this didn’t work the same for all people. While people who used more back-steps generally did worse in debugging, people with instructor-directed values did particularly worse.
So, with those results, here are my main take-aways:
People in different cultures use back-steps differently. We saw that some of this can be explained with self-directed learning, more specifically self-directed learners take more back-steps. We also can see that some differences, like those between the UK and Australia, are not explained by differences self-directed learning.
Back-steps in debugging may not be helpful. We can’t tell from our experiment why more back-steps were generally correlated with worse debugging. It could be that using back-steps is actually detrimental. It could also be that people who are struggling use back-steps more as they flail around. We need more experiments to find out the true cause of these results.
The benefits of back-steps may vary with culture. We showed that instructor-directed learners did particularly bad when using many back-steps. We don’t know exactly what is going on here, but this result points to how the benefits and detriments of back-steps might vary with different groups of people.
To make programming tools that benefit everyone equally, designers need to consider culture. People around the world will make different uses of their tools. For example, creators of debuggers with back-steps (or other non-linear navigation) might want to vary how back-steps are emphasized, explained or displayed for different users, or they may want to find entirely different ways of presenting the information programmers need.
If we don’t consider culture, the people with the most influence may assume everyone acts like them and they may build tools that primarily benefit them. Most programmers don’t live in Silicon Valley, so let’s make sure we design our programming tools for people all around the world.
More Information
The research paper I wrote was peer reviewed and accepted at the 2018 IEEE Symposium on Visual Languages and Human-Centric Computing. I presented these results in October in Lisbon, Portugal.
The Impact of Culture on Learner Behavior in Visual Debuggers. Kyle Thayer, Philip Guo, and Katharina Reinecke (2018). IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).
You can contact me at kthayer@cs.washington.edu or visit my website http://www.kylethayer.com.
Previous Post: Coding Bootcamps: Women Overcoming Intimidation
Next Post: Measuring Values and Culture
Footnotes
1. The following are some examples of prior research on culture, student preferences, and navigation:
- G. Hofstede, “Cultural differences in teaching and learning,” International Journal of Intercultural Relations, vol. 10, no. 3, pp. 301–320, Jan. 1986.
- P. J. Guo and K. Reinecke, “Demographic differences in how students navigate through MOOCs,” in Proceedings of the first ACM conference on Learning@ scale conference. ACM, 2014, pp. 21–30.
- A. Marcus and E. W. Gould, “Crosscurrents: cultural dimensions and global Web user-interface design,” interactions, vol. 7, no. 4, pp. 32–46,2000.
2. For more about Power-Distance Index, see Wikipedia and:
- G. Hofstede, Culture’s Consequences: International Differences in Work-Related Values. SAGE, Jan. 1984.
- G. Hofstede, “Cultural differences in teaching and learning,” International Journal of Intercultural Relations, vol. 10, no. 3, pp. 301–320, Jan. 1986.
