Negative Potential in CS
I read, with great interest, the suggestion on disclosure of positive and negative impacts in papers put out by the FCA. The gist was an argument for a, “section [that] would summarize both the anticipated positive and negative impacts of the paper and motivate these anticipated impacts with the proper citations” and that reviewers should apply this by “leverag[ing] the gatekeeping functionality of the peer review process.” I’m delighted that the FCA articulated a concrete set of goals in the post (which I largely agree with) and is tackling the problem of impacts directly. I would very much like to see an array of proposals to this effect. Since the post was intended to, “start a conversation rather than end one,” if you will allow me…
My first reaction was: “You want a bigger discussion section? Sure, if you give me another page of space.” My second reaction was: “nope, probably not even then.”
Here’s the TL;DR: There is a time and a place for nuanced arguments, but I’m going to go Kieran Healy on this one and argue that it doesn’t belong in most papers, and certainly not for gatekeeping (see, I read social scientists). Furthermore, I think there will be unintended consequences that make the quality of our discussion on the role of CS worse, not better.
I largely agree with the post’s outcomes as being good and desirable, but disagree with them being “expected” given the intervention. I’m not convinced that “this simple change [in reviewing],” will, “achieve substantial change.” The proposal makes it sound that we should expect a positive outcome, or at the very worst, a neutral one. To me, there are potential downsides that are unfairly dismissed.
First, I fear authors will not produce thoughtful discussions and we’ll get something between speculative fiction and lip service. I also believe that reviewers are incapable of evaluating such discussions in a principled way, and I’m leery of introducing more subjectivity into the process. Perhaps most important, is that I don’t think it will help in the places we really need to be making a difference. Consumers of our research — companies, the press, policy makers, even “the public” — will not likely benefit from discussions in this context. For authors, this kind of “feedback” will come much too late to make a difference (the research is already done!). What I do believe is there should be more spaces for critique, discussion and, better yet, proposals for solutions — but that the place for this is not in standard research papers.
One of the most troubling suggestions of the post is that reviewers should just go ahead apply the guidelines and cite the post as justification. If a community decides they want to implement this — with notified authors and trained reviewers — so be it. However, as an AC/SPC, I would not look kindly on a reviewer implementing this unilaterally, and would likely dismiss their review if they did. A bad execution of this “just do it” strategy may be especially damaging to junior reviewers.
As many of my co-committee members can likely attest to, I’m not a believer in the “literature will work it out” argument. Organizations who publish are providing an endorsement of not only a particular piece of work but of a way of doing work. We have, and will continue to push back on papers that have ethical problems or fail to disclose the damaging implications of their work. We’ve argued (successfully) for rejection of papers that are likely to have negative impacts. We would have done this whether the authors “mea culpaed” or not.
Speculative Fiction
Let’s go back to what we might see authors producing. Specifically, how the author will communicate implications. To me, design implication sections always read like “speculative fiction.” Anything that the author had actual evidence for probably made it into the results or discussion sections of the paper. I think reviewers are intellectually aware of this and tend not to have papers accepted or rejected based on design implications (some would even have us avoid this section entirely). Sure, if an author overclaims they’re likely to get some pushback, but we try not to accept or reject based on this.
I’m sincerely uncomfortable with the notion of authors and reviewers using meta-discussions of the future to decide the fate of a paper.
#31
And maybe what we’ll see won’t be speculative. Maybe it’ll be boring.
There’s a joke I like about this elderly couple. The husband and wife are reading a newspaper at the coffee shop. The old man looks up at his wife and says, “number 25!” She looks up and smiles and goes back to reading her newspaper. A few minutes later, she looks up and says “number 31!” to which the old man smiles and goes back to his reading.
A young woman watching this exchange asks the barista, “do you know what’s going on over there?”
“Yes,” he says, “they’ve told each other the same jokes for 30 years. Now they just say the number to save time.”
The problem with repeating the same lines over and over again is that they really lose effectiveness when we’ve heard them all before. There was a period where every human computation talk I’d go to would be invariably followed by a question from the audience about the implications in the long term for the workers. “Aren’t we just making it worse for workers?” or some variant of the question would get asked. The speaker would smile, shrug, make some conciliatory indication that they understood the problem and that there’s research on how to make sure Turkers also learn from the tasks and get a fair wage. And that would be the end of it. An exasperated speaker and an unsatisfied questioner repeating the same dance. Take as another example this from the vision community (see the “what it means?” section). It’s engaging (and funny) the first time. It won’t work the 50th.
The danger, of course, is that over time — even for well meaning people — the questions and answers become meaningless. A ritual where no one actually engages with the problem because neither the questioner nor speaker have an idea of the problem’s significance or how to solve it. They simply signal to each other that they are on the same page and that we might have (or might be!) a problem.
Let me hypothesize that there are a limited number of societal “problems” that any particular technology might cause: loss of privacy, agency, social mobility, social justice, the many *isms, etc. All these are completely valid concerns. But I can just envision papers ending with: Results in this work have potentially negative consequences to issues #4, #27, and #59 and we propose using mitigating approaches #23, and #31. Do we really want our discussions reduced to this? Can we expect that they’ll be more than that?
The Goals
Let me try to work through my (re-interpretation) of the mid- and long-term goals of the proposal.
Making (the right) problems salient
There’s something I call the Prop 65 problem. In California you will run across a bunch of labels like this: “This [product|store|place|etc.] contains chemicals known to the State of California to cause cancer and birth defects or other reproductive harm.” This is due to Proposition 65 and the labels are everywhere. The problems with — what started as a well meaning law — are that (1) it is somewhat vague (which has led to coffee having prop 65 warnings due to acrylmaide in coffee) and (2) because it’s often safer (from a legal perspective) and easier to be proactive, companies will put up the signs even when they’re not necessary (there’s no consequence for over warning). The consequence is that prop 65 warnings are everywhere in California, whether they are appropriate or not.
We are likely to produce the same kind of white noise in our papers — something to tune out. No matter how well meaning we are, the rigor and intentionality in which authors and reviewers engage with this will not be particularly high. Further, given the incentive this kind of disclosure policy might create, authors will proactively claim unrealistic problems. And why not? A reviewer can’t accuse me of failing to acknowledge a problem if I acknowledge all problems. The time and place in which we discuss important issues should not become so ritualized to the point where (a) the issue becomes easy to ignore, (b) it is impossible to tell whether the issue is real.
So if it’s not the individual paper, where should this discussion be? What’s more likely, to me, is that a significant line of research or combination of ideas will result in both positive and negative impacts — just not the ones we anticipated from the atomic pieces. Given this, I’d prefer that instead of making space for each individual author to try and overestimate their impact on the world, that we give space to people who want to engage in deeper thinking about how systems interoperate to cause benefit and harm. Maybe it’s letters to the editor, maybe it’s survey articles, maybe it’s more panels, maybe it’s new venues. I’m not sure what form(s) this will take in the end, but we should treat this like any other problem of developing good ideas and making them (persistently) visible and not easily ignored.
Making the problems salient to the (right) people
Maybe the hype cycle in the press or elsewhere is a legitimate concern. In that case, there are lots of ways of controlling the conversation with the public, but embedding cryptic (academicese) messages in papers isn’t high on the list. I think we need to correctly model journalist incentives in this discussion, as they relate to our goals and where, and how, our messaging will best work within the structures of the press. For example, while I have mixed feelings about academics as public intellectuals, I’m pretty confident that style of work does more than an implications section.
The media flourishes in the presence of controversy: the new versus the old, the nays versus the yeas. That, “tech stories of late frequently already adopt the framing that we suggest above” should not be a surprise. The media also enjoyed reporting on vaccines and autism (speaking of unintended consequences and mea culpas). Again, this not to say that the tech press doesn’t serve an important role or isn’t doing a good job. They do, and they have. But once a story is out, our ability to control it is gone (this based on some personal experience, see question #10). All I’m saying is (1) don’t expect the press to save us or create the right incentives, (2) be careful what you wish for, (3) get researchers media training.
If the fear is that somehow we are training students to be bad actors, then by all means, we should add a significant ethics curriculum. Students — those who become academic and those who work for industry or government — can (and should) be trained to ask critical questions. However, the place for that is not in the discussion section of papers.
More Examples Please
Let me end with this: I’m willing to be convinced that the proposal is a good idea (if not now, perhaps in the future). But, I’d like to see more examples of the problems that this approach would help mitigate and the benefits that would emerge. There is a very limited set of technologies that I can imagine that are wholly positive. Such technologies tend to be extremely targeted (no broad technology can be unambiguously good) and individual-focused (no social technology can be unambiguously good). I’m very happy for these to be funded, but they are a very small part of what we can and should do.
Everything else is messy. I’m fairly confident that we have very few people, if any, in the academic world who, “continuously [choose] to conduct anti-social research.” In part, because I don’t think we can agree on what that looks like. If we can’t agree either on what technology tilts pro-social, what mitigates anti-social tech, or even what unambiguously (and persistently) anti-social tech looks like, perhaps we don’t jump to making reviews more subjective quite yet?
Maybe at some point we’ll be able to agree on which ideas are more good than bad but that’s a long conversation. A conversation that I don’t believe can or should be short-circuited by reviewers making gatekeeping calls on paper implications sections.
End note: I feel like I need to apologize in advance to my friends and colleagues at the FCA. The tone is to get people to read a very long post, and not to be reductive. The authors of the FCA post are well-meaning and thoughtful and we clearly share the belief that there is a problem in how we impact the world.
Thanks to my nice friends who gave me feedback on this. I’m not sure they want to be associated with it though, so I won’t name them.
