How to Revolutionize Science by Gaming: Introducing COSMOS

Computer games can do way more than entertain: they can be designed to simulate all kinds of scenarios. This makes them a perfect tool to research human behavior. Games can propel psychology and all other disciplines that depend on measuring human traits or skills. From cognition to social fairness: the virtual sky is the limit!

“Play is the highest form of research.” — Albert Einstein

Measuring human behavior

One of the core tasks in psychology is to observe how individuals behave. We call this measuring, which is nothing but the assignment of a score that should represent some characteristic of an individual. Whether you’re after aptitude, the effect of a therapeutic intervention or assessing the outcome of an experiment — typically, you rely on observations that you try to standardize and thus make quantifiable. That’s psychometrics. Of course, in some cases, this also includes self-reports of individuals, whom you trust to be capable of observing their own behavior.

Psychologists have come up with an enormous amount of instruments to measure, e.g., IQ, personality traits or prosocial behavior. Those instruments are very diverse and span from experimental setups over questionnaires to testing batteries. Still, they have one thing in common: they all were designed to measure something as accurately and reliably as possible. Whether the testee was having a good time while undergoing these test was never a guiding design principle.

This needs to change!

Let me tell you why and how.

Online games can serve as psychological testing instruments and you can help advance science by playing them.

Combining psychology and genetics

15 years ago, when I was a young undergraduate student in psychology, I started to work in a behavioral genetics lab. I was convinced (and still am) that combining psychology and genetics could provide a window into the biology underlying human behavior. After all, everything we do and think and all the memories we store, require a vast multitude of proteins and chemical processes to work seamlessly together. This is the very essence of what is making up the most complex molecular machinery we know: our brain.

Measuring behavior and correlating it with differences in our DNA can point us to the molecules that are necessary for the cognitive processes we are interested in. If you want to figure out the molecular pathway that underlies the formation of memories, you first have to test memory performance in a large group of individuals. You will end up with a distribution: some will show excellent memory, some will hardly remember anything, and the bulk of people will end up scattered around the mean. You then scan the DNA for differences and statistically check whether some of the variants are more common on one end of the distribution than on the other.

If you are lucky and can come up with a robust finding that points to a certain gene, you can call your molecular biology colleagues. You tell them about your finding and ask them whether they can figure out what the protein that your gene encodes is doing. All you do is pointing them to a protein based on your statistical analysis they probably wouldn’t have investigated by themselves. If all goes well, you have identified a piece of the huge puzzle, how the brain stores information.

But wait, what does all this have to do with online gaming?

There’s a huge issue with this approach. The effects of single genetic variants that you are looking for are tiny. Incredibly tiny. This means you need to test a huge amount of individuals before it makes any sense to start doing the costly DNA analysis part. There is one shortcut though: You can also just look at extreme groups. Effects should be bigger in this case, which means you need to look at fewer DNA samples. Still, to identify enough individuals exhibiting extreme traits, you need to screen a very large population first. So what do you do, if you plan to assess thousands of people? Preferably you want to do so even repeatedly, as averaging over several measurements gives you a better estimate than a single one-shot measurement. This is where the games come in.

Given the task of psychometrically testing thousands of individuals, it was obvious we had to come up with a solution that would automize data collection. We also quickly realized that using online games instead of conventional psychological testing in our lab could very well do the trick. There’s a whole series of advantages to this approach. To name a few: You don’t have to deal with the logistics of bringing people into the lab. You can test worldwide, not only the WEIRD people (Western, Educated, Industrialized, Rich, and Democratic). You can pay people back with entertainment and by telling them the results generated by the testing games. You can also give immediate feedback, because you record everything in a database and thus scoring can be done automatically (data entry that may otherwise be required, if analogous tests are used, is not only a tedious task but also an optimal source of error).

Facing the replication crisis

While we, of course, set out with genetics in mind, we soon realized that games could serve all sciences that investigate human behavior. This becomes important in the context of the huge replication crisis we are currently facing in psychology and neuroscience. While non-replicating findings are still shaking up the field and a ferocious debate about the topic is going on, the solution is trivial. We need to improve the quality and quantity of observations to allow more valid conclusions. This means we should: A) have more data per individual and B) survey more individuals than the average study currently looks at. A) can be achieved by measuring repeatedly and by also measuring constructs or behavior that influence the measurement of interest. Some correlations may depend on the particular subgroup you’re looking at. B) can be achieved by changing the way we recruit. The COSMOS platform (which by the way is an acronym for COgnitive Science Metrics Online Survey) is one idea to deal with the latter.

Brain training

And there’s one more thing. Brain training. Can you actually improve your cognitive skills by regularly training with cognitively demanding games? According to a recent market research report, the cognitive assessment and brain training market size will grow from its 2 Billion USD in 2016 to a forecasted 8 Billion USD in 2021. This is quite an impressing price tag, given that the jury is still out on the efficacy of brain training. It may well be that only a given subgroup like the elderly benefit from gamified cognitive training. It could also be the case that the gained improvements are independent of the exact type of video game played. Or on the other hand, need to have specific features to work. Either way, we should figure out what cognition games can do and what they cannot do. And I’d feel a lot better if this was to be figured out by scientists that are interested in solving this question and who are not making money from selling the product.

The additional upside of this for everyone: You get the brain training for free! We’re scientists that are interested in advancing science, not in selling you stuff. Our goal is to base scientific findings on the best possible data foundations. This is well worth mentioning because this is what sets us aside from for-profit organizations. We’re committed to the ideals of Open Science, Open Data and Citizen Science, as far as the privacy of every participant can be guaranteed, of course. We would never share data that has not been completely anonymized. It’s self-understanding that the intended use of the data will always be for research purposes only.

Cooperation goes a long way

Of course, programming games and putting them on a secure online platform is a little bit easier in theory than in practice. When we started, we imagined the whole endeavor to be way easier than it turned out. But I learned a few things over the years hopping between different research disciplines. It’s amazing what can be achieved if you combine the skills of different fields of research.

During my PhD time, I learned a lot about the methods we use to measure DNA variation. I was flabbergasted, how tightly different disciplines like biology, chemistry, and engineering needed to work together to generate the complex methods we now use to read DNA. Not to mention the IT developments that needed to be made to allow handling and storing the enormous data sets that were created by these high-throughput methods in the lab. The experience gathered didn’t only teach me how to deal with big data sets, it was also a lesson in what you can achieve if you cooperate.

When we couldn’t get any funding, we realized that universities should have all the skillsets in place that we need. So we reached out to the informatics department of the University of Applied Sciences Northwestern Switzerland. All the games we have developed so far have been programmed in the scope of student projects. (Big, big thanks to everyone involved!)

Let’s do this!

COSMOS is currently still in its pilot phase. We now need you to really make it work! We will keep adding more games (measuring, e.g., also fairness, humor and theory of mind) and refine the platform in the coming months. And of course: We are always open for cooperation if you are a fellow scientist!

I think it’s about time more researchers start to realize the potential of computer games for observing behavior. Psychology, economics, political sciences, sociology, health science. To all behavioral science disciplines: Let’s think about how we can capitalize on games to get our research done! Let’s bring some innovation to science! Let’s do this!

Register an account on COSMOS and check out our games at cosmos.psycho.unibas.ch

Follow us on twitter for news and updates.

To show your support for this post and recommend it to your followers, click on the clap icon 👏 below. Each users is allowed to clap up to 50 times to show how much they appreciated a story.

The University of Basel has an international reputation of outstanding achievements in research and teaching. Founded in 1460, the University of Basel is the oldest university in Switzerland and has a history of success going back over 550 years. Learn more