“How do I know this is real?” Research and the Context of ‘Cred’

Jeremy Birnholtz
Jul 31, 2017 · 8 min read

“How do I know this is real?”

So said a 16-year-old potential research participant when I reached out to him via Instagram direct message about participating in an interview study (1). It was a smart thing to ask, but also — as a tenured professor at an elite university — not something I get asked very often in doing my work. I was messaging him from a lab account (@nusocialmedialab) that mentioned Northwestern, linked to our lab web site and mentioned our reasonably-subscribed Twitter account. All of this, I (naively?) assumed would convey some credibility.

Image for post
Image for post
My lab Instagram profile. Note very small number of posts, small set of followers, and terrible follower:followee ratio.

I was wrong. As this young man and several other prospective participants have pointed out, the Instagram account had all the marks of sketchiness. It was a recently created account with only one post (at the time) that in turn had almost no likes, it had a very small follower count (still < 100), and it was following a pretty small number of others. Who was I in this community and why did I merit their attention? Well, mostly I was nobody to them.

To be sure, when people asked I was usually able to get around these issues by sending more detailed study information and inviting them to contact Northwestern’s IRB with our study number. Lots of people probably didn’t bother to ask, though, and I’ll never get to talk to them. More importantly, though, this experience got me thinking about “cred” (urban dictionary: “short for ‘credibility’. An ability to inspire belief in others.”) and its context-dependence.

As researchers, we — like any community comprised of humans — have our own value system and ways of conveying ‘cred.’ Some of these, like tenure, keynote invites, citation counts, zillions of grant dollars and Science articles apply across many fields, while others, like CHI or CSCW papers, apply only in specific fields. For obvious reasons related to our own survival, we often fixate on these. We publish lots of papers and wish we could be like the people who publish more than we do. We furtively look at our citation counts on Google Scholar and compare them to our peers. We expect to be listened to and respected after we’ve hit certain levels on these metrics.

Image for post
Image for post
Check it out — people cite my work!

These are important within the research community (though, yes, they have drawbacks — that’s a separate conversation), but I also believe that it’s critical that we as a research community also recognize the boundaries of these value structures and their limitations. If we don’t recognize these limitations, we threaten our cred where it matters most — in the communities (and world more broadly) we’re trying to study and impact.

This isn’t just about the communities we study. It’s also about the tech industry that designs and builds the technologies we study, and I think these phenomena are related. I spent time (6 months in 2015; 3 months in 2016) over the past couple years as a visiting professor doing research at a tech firm. On balance this was an amazing experience in which I was challenged and learned more than almost any other time in my recent memory. But it was also humbling in some important ways related to cred.

First, I (like many Ph D’s visiting and being hired by tech firms) assumed that my academic cred meant that I would get some automatic respect in meetings and that people might listen to me. Wrong. It’s not that people were disrespectful (they weren’t!), but rather that they clearly weren’t handicapping my score because I came in with a CV full of papers. (This expectation may sound self-aggrandizing and I don’t think I consciously expected it, but reflection has revealed to me that it happens all the time in meetings with new collaborators or guest speakers.) I had to earn respect from proverbial square one, by showing that I understood and possessed some local markers of cred, such as the ability to talk semi-coherently with engineers and subtly conveying awareness that everything is different “at scale” (aka recognizing the limits of my extant knowledge in this environment where people believe none of that applies).

Second, I had to realize that, as a researcher, my voice was just one of many around a table where there were also engineers, designers, product managers, communication specialists and others. This is at pretty direct odds with the fact that, as scientists, we are often socialized to accord primacy to the scientific truths that we seek. There’s good reason for that, but this self-accorded primacy doesn’t change the political reality in societies and organizations that decisions are informed by many perspectives. Persuading others that our perspective is important (and hopefully correct :) ) needs to involve more than just laying out the empirical findings. We also need to, as I learned and watched others learn, understand the perspectives of others at the table, draw on these perspectives and find useful points of overlap. For me, this meant, for example, understanding the origins of others’ beliefs, talking about why a product was designed in a particular way or asking how a sequence of decisions led to a current situation.

Image for post
Image for post
Excerpt from the Instagram profile of a gay 17-year-old with much more local cred than me.

If cred is measured in followers (as it is obsessively and formally tallied by my Instagram participants and informally measured within said tech firm), I don’t think I earned a ton of it in my time at this firm. My final presentation of my work there was pretty sparsely attended (in comparison with other presentations by those I know have a lot more local cred) and I still got some blank stares when I met new people and told them what I was doing there. I do believe, though, that I earned respect from people I sat in meetings with. Sometimes this happened as I shared results that directly addressed a perspective somebody had brought up (e.g., “I understand that you believe that, but here’s why I don’t think your understanding is correct or will lead down the right path”). Other times it was because I had demonstrated some level of competence that seemed to deem me worthy of listening to in the future (e.g., “Right, but couldn’t you solve that problem by just doing x, y and z?”). I’m guessing at other people’s perceptions here, of course, but that was my impression.

In addressing the challenge of cred in the communities we study, there’s a long-running discussion about insider/outsider status in populations and ways that researchers can gain credibility with informants.These are important topics to be sure, but what I’m really concerned about here is just having enough cred to be able to do something like recruit participants for a study and distinguish ourselves from trolls, spammers, catfishers and other nefarious characters, especially in environments where we can’t fall back on our traditional markers.

One key data source for my dissertation was a mail survey of researchers in three fields. To increase the chance of response, I got (what I’d now call) “shoutout” letters from a prominent researcher in each field, encouraging their peers to respond to my survey. This could be one way to get some instant cred, though it begs the question of how to approach somebody for a shoutout in the first place.

As another example, I’m in the early stages of a project looking at how gay men in India use dating and hookup apps. There’s some legal and social risk in outing oneself in India even as a participant in research, so one suggestion I’ve gotten is to make clear to potential participants that I’m gay to make the study seem, I guess, less threatening. That’s fine on the one hand, but it’s also true that I’m not gay and Indian. So will they really relate?

Similarly, I’m currently studying gay/bi teens and young adults on Instagram. I could again reveal that I’m a gay researcher, but I’m certainly not a teen or young adult. On the one hand, being out as gay might help. On the other, would I be less threatening if I were straight and therefore (in their minds; I’m not considering this) less likely to be “creepy” in an interview? Unclear. Or would it help if I (somehow) built up a large follower count, and then revealed that I was a researcher seeking participants? Again, unclear, but there’s also some chance that a strategy like this might backfire or even be seen as unethical if people felt like they had been ‘led on’.

Another thing, building on my tech firm experience, that I think this means we need to think about as an HCI/CSCW research community that purports to effectively straddle academia and industry is our cred on the outside. If our work isn’t taken seriously by the firms we purport to help with our work or by society more broadly (which I haven’t really addressed here, but is sometimes true), should we try to change that? And if so, how do we do that? As Mary Gray pointed out in a recent Chronicle piece, it’s important for us to collaborate with industry researchers, understand their problems and be able to work with their data. Another reason this is important is that it can be easier to make an argument to a firm when you’re working with their own data. By the same token, though, it can be tricky to negotiate what it is acceptable to publish and put out into the world beyond these firms. How one negotiates that boundary can, in turn, affect one’s cred back in the research world, as Mary rightly points out. Some caution is required, but running away is not the answer.

Finally, I think it’s imperative to at least be conscious of the value and cred structures we inhabit, and recognize when we’re crossing the boundaries of that system. This can help foster a recognition of various perspectives brought to bear in decision making and trying to understand why other people, however irrationally or foolishly in some cases, believe the things that they do. It’s rarely because they’re genuinely stupid, and it’s much easier to have constructive dialogue with somebody who believes you’re taking them seriously, even if you disagree with them.

  1. Yes, all of this is consistent with my approved IRB protocol.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store