Misadventures in Data Collection: How the Social Observatory came to be

Before there was a Social Observatory, there was still social experimentation.

But without the facilities and tools we have now, it went hilariously awry. This is the story of the best laid plans of mice, men, and social experimenters

It all started when…

The Social Observatory team TOOK OVER a bar on a Friday night. A real life bar, and without the knowledge of the patrons

We partnered with the Romper Room, a bar in downtown San Francisco, to ran a controlled experiment. Our goal was to understand more about how humans behave at bars and try to intervene in a way that could increase connection between strangers.

The experiment

If you really just wanted to hang out with your friends, would you really choose a bar? Wouldn’t a better spot be a place that you didn’t have to shout to hear one another, a place that was easy to find a seat and drinks were much cheaper? Of course.

Of course people go to bars to meet new people. This is the implicit purpose of a bar.

But it is difficult.

Even with the best self talk, men and women freeze up. What to talk about? What if they say no? And thus, more often than not, we go to bars with the intent to meet new people but end up just hanging out with our friends.

This age old problem has not, to our knowledge, been solved. New mobile dating apps like Tinder, Hinge and Bumble help us avoid relying on bars as our sole funnel for dating, but they don’t make meeting people — when in person — any easier.

The Social Observatory team’s take over of the Romper Room attempted to solve this problem. We designed an 2 condition experiment with hopes of figuring out how to be the social lubrication that bars desperately need.

After people were stopped at the entrance and asked to fill out our survey, we assigned them to a condition. If you were randomly assigned to the downstairs bar, it was a normal bar night for you. You were in the control condition.

However, if you were assigned to the upstairs bar, you had a very different experience. There was a “greeter’ who would be your concierge for the evening and introduce you to the right people at the right time.

Did it work?

Did the Greeter intervention help to the grease the wheels of social interaction more than the control condition? I CAN’T WAIT TO FIND OUT!!!

We have no clue. Running an experiment in a bar, at night, with people running all over the place, drinking — is very very difficult.

It’s not that we weren’t prepared. Oh no. We were like the Ant teaching the Grasshopper a lesson of prepardness. There was a required practice session for us the weekend before. We had 8 trained volunteers at the event helping with logistics.

And yet colossal failures happened at multiple points of weakness.

Survey compliance
Who goes to a bar and gets asked to take a survey prior to entering? This is not only abnormal it’s a bit suspect and questionable.

Random Assignment
Random assignment is hard when people travel in groups. John shows up to to meet his friend Laura. John was assigned upstairs but Laura is downstairs. John and Laura are now in both conditions.

Drunk people don’t want to take surveys
It was near impossible to get people leaving the bar to indulge in another 5 minute post — survey. And arguing with a drink person is unadvisable — even for science.

Data data data
We had an intricate system to record who was in which condition. Like most things in a bar, it got sloppy and by the time people left we weren’t sure which end up was up.

What was abundantly clear from this experience was that it may be easier than we think to help people meet others at a bar. From our biased perspective — the intervention worked, but we definitely don’t have data to prove it. Designing and execute a randomized controlled experiment on (drunk) humans, in the wild is hard.

And hence the observatory

This is essentially why Social Observatory, in it’s current form, came to be. It is insightful but not sufficient to experiment on human behavior in an academic lab. However, the alternative, testing on humans in the wild, is very difficult without a system to support rigorous experimentation.

Social Observatory aims to make it easier to get data about how humans behavior in the wild by making the process of recruiting subjects and collecting data before, during and after the event stupidly simple. We will build and evaluate new methods and tools to advance rigorous, reproducible social science studies at scales necessary to develop and validate causal models of human social behaviors.

Instead of having to argue with people to take surveys, the Social Observatory events have people excited to socialize for science. Instead of doing paper surveys with intricate data systems, all attendees have downloaded an app and get push notifications at the right times.

And of course, if participants drink too much, we’ll know. The space where our events are hosted allows us to serve alcohol — which means we can track how many drinks each person gets to know if/when to remove their data.

In the future we can only hope that more people are Socializing for Science vs just socializing at the bars.