Bridging the Lab and the Field
New Challenges for Social Media Experiments — And a Workshop!
The 2017 ICWSM keynote talk by Matthew Salganik described a critical gap in the study of social behavior. While many researchers are still focused on conducting surveys or traditional lab studies, others are focused on analysis of “big data”. Unfortunately, neither of these methods provides a full picture of online life and behavior.
Up until recently, most behavioral studies were carefully designed experiments created to explore specific hypotheses or research questions. With enough care to experimental design and data analysis, researchers can establish causal relationships between behaviors and their outcomes, and easily replicate their results. These experiments have high internal validity: they’re controlled enough that researchers (usually) know why they’re getting the results that they get. However, these studies are often lacking in ecological validity, as people who are asked to do tasks in a lab generally know that they are in fact in a lab. This limits the generalizability of these findings.
In recent years, the availability of big data — or “found data” — has changed the game for many researchers. They collect and analyze large amounts of observational data from online platforms, using the traces of our conversations, browsing habits, and other behavior to understand many complex issues. These analyses powerful because of the number of data points involved, and they’re useful for capturing what people do in real-life online situations. However, the lack of control over data collection makes discovering causal relationships extremely difficult, and limits researchers’ ability to replicate their results.
So how do we bridge the gap between lab data and big data? The Social Media Lab’s solution is to use the capabilities of the digital age to create new forms of “designed data”. Our own research projects are attempting to bridge the lab and the field by using a simulated social media site to conduct studies. The realistic social media environment lets us capture natural interactions, while also maintaining careful controls that allow us to establish causality. Plus, the code to create and run your own version of the tool is available online, making our studies easily replicable by other researchers.
While we’re excited about the potential of this research method, we also know it isn’t the only interesting new method of data collection out there. So, we’re putting together a workshop at ICWSM 2018 to explore novel ways of thinking and building social media behavior experiments in the research community.
The workshop will showcase position papers that explore more naturalistic experimental paradigms, from field-like lab studies to lab-like field studies to pure field studies. We also want to hear about innovative tools, methods, and challenges in conducting research with both higher internal and ecological validity. Workshop participants will brainstorm challenges of social media experiments and solutions to these challenges. For example, we will discuss how researchers can appropriate existing digital environments, as well as create new digital platforms and tools to test theories and answer fundamental questions about human behavior and technology.
By bringing together social scientists and designers, the workshop will foster an interdisciplinary dialogue and new research collaborations.
You can find out more about the workshop, including instructions for submitting work, on its website. We look forward to hearing your ideas!
