Wikipedia, controversy, and an acclaimed documentary

The Hunting Ground, a 2015 documentary about sexual assault on college campuses exposed conflicts of interest, malfeasance and cover-ups.

Drawing by Nicholas Boudreau, licensed CC BY 4.0.

To learn about a complex topic — especially if powerful institutions have a major stake in it — we rely on experts. People who devote substantial effort toward understanding all facets of a topic can offer the public a great deal of value. We routinely refer to their perspectives and analysis when forming opinions on important social and political issues.

But of course, our reliance on experts makes their interests and motivations highly significant. To what extent do an expert’s motivations inappropriately drive their opinions and judgments? Do those opinions and judgments color how they present the facts? As critical readers, we should always pay attention to conflicts of interest (COIs). And if they’re insufficiently disclosed, we’re at a significant disadvantage. If you learn that a product review you relied on was secretly written by the company that made it, you might feel some indignation — and rightly so.

Publishers that care about accurate information face the same issue, but have a greater degree of responsibility; and if a publisher inadvertently amplifies biased information on to its readers, its reputation may suffer. So publishers establish standards and processes to eliminate COIs, or — since it’s often impossible to gather information that is 100% free of COI — to manage them responsibly. Wikipedia is no exception.

But as a publication that invites participation from any anonymous person in the world, Wikipedia has unique challenges around COI. Despite Wikipedia’s efforts to require disclosure, COIs often go undeclared and unnoticed, which leaves everybody (understandably) a little skittish about the whole topic. Blogging pioneer Dave Winer’s words in 2005 illustrate this point: “Every fact in [Wikipedia] must be considered partisan, written by someone with a conflict of interest,” he said. But significantly, every change to the site is rigorously preserved and open to public review. Wikipedia editors routinely investigate and deliberate additions and edits to the site. Every change can be reverted. Every user can be chastised or blocked for bad behavior. The process can be messy, but since 2005, researchers have repeatedly found that Wikipedia’s process generates good content. A properly disclosed and diligently managed COI on Wikipedia is rarely a big deal; it’s part of what makes Wikipedia work. Disclosure is a key component that supports informed deliberation. Disclosing a COI doesn’t give a Wikipedia user carte blanche to do as they see fit; but it does express respect for Wikipedia’s values and for fellow editors, and it gives Wikipedia editors more information to use in resolving disagreements.

One of the principle methods Wikipedia employs to minimize the impact of COI is an insistence on high quality sourcing. But on occasion, Wikipedia editors are overly swayed by sources that match up poorly against the site’s standards.

See our previous blog post, Conflict of interest and expertise, for a deeper look at the subject.

A Wikipedia case study

The Hunting Ground (2015), a documentary film which investigated the issue of sexual assault on U.S. college campuses, received widespread acclaim, but it also ignited controversy. The production company, Chain Camera Pictures, retained Wiki Strategies beginning early that year to assist with developing and improving the Wikipedia articles related to the film’s focus, as well as the article about the film itself. (See “Disclaimer” below.)

Conflict of interest is a central focus of The Hunting Ground. Universities are required to investigate any report of a sexual assault involving their students; but they also have a strong financial and reputational interest in avoiding scandal. By vigorously investigating sexual assault cases, universities might associate their campuses with violent crime, which could impact recruitment and alumni donations.

In one of the incidents explored in The Hunting Ground, Florida State University (FSU) football star Jameis Winston was accused of rape. A state attorney, when announcing months later that he had insufficient evidence to prosecute, noted substantial problems in the initial rape investigation carried out by both FSU officials and Tallahassee police. Independent investigative pieces from Fox Sports and the New York Times both suggested that COI might have been a factor.

While the influence of a COI in any specific case is difficult to prove, it’s clear that the financial interests of entities like FSU―whose athletics programs bring in more than $100 million a year―sharply conflict with the interests of the women portrayed in The Hunting Ground. FSU is one of the many institutions that had reason to feel threatened by the film, alongside numerous universities, law enforcement agencies, and athletic programs.

The Hunting Ground earned substantial accolades and validation. It received two Emmy nominations, including Exceptional Merit in Documentary Filmmaking, and was one of 15 documentaries shortlisted for the Academy Award for Best Documentary Feature. CNN vetted and broadcast the film, along with a series of panel discussions, putting its own journalism reputation on the line. And student groups, faculty, and university administrators screened the film on hundreds of college campuses.

But unsurprisingly, given the threat it posed to powerful institutions, the film drew pushback as well as praise.

Among the film’s more persistent critics has been the Washington Examiner’s Ashe Schow, who has written columns about it or mentioning it more than 20 times since March 2015. In November 2015, during the runup to the Academy Awards, Schow announced Chain Camera’s Wikipedia efforts, under a headline proclaiming that they had been “caught” editing Wikipedia. But of course, you can’t be “caught” doing something you were open about from the start; and Chain Camera had been diligent about disclosure. Wikipedia editors working on the various articles had known of the efforts of Chain Camera’s employee Edward Alva for many months. As Chain Camera stated in their rebuttal, Schow’s charges were inaccurate and ill-informed.

Wikipedia co-founder Jimmy Wales took Schow’s words at face value, praising her piece for “embarrassing” Alva. As Wikipedia editors took up the issue in a public discussion, several included Wales’ statement as part of the evidence of Alva’s wrongdoing. Much of the early discussion was characterized by a lack of diligence in considering Alva’s efforts. One comment stands out: “I don’t have enough time to do a thorough investigation,” said a Wikipedia editor. “But as I now see it, this situation could be dealt with very quickly and justly with a permanent ban of [Alva].” A lack of thorough information was apparently not enough to stop this editor from recommending strong sanctions.

The complaints about Alva’s editing, made initially by Schow and amplified by Wales, were considered in detail. The graphic highlights the formal decision by administrator Drmies. Full discussion.

But as many Wikipedians recognize, diligent investigation is important. In the following week, several Wikipedians did indeed take a close look at the edit history. They ultimately rejected the accusations leveled by Schow and Wales. Drmies, the administrator who made the formal determination to close the discussion, stated that “the [Chain Camera] editor declared their COI early enough,” and that “the editor’s defenders present very strong evidence that [Wikipedia’s] system worked.”

Drmies’ closing statement carries weight in the Wikipedia world, and it is archived publicly. But from an outside perspective, it might as well be invisible. The media world is used to covering traditional decision-making processes, like court decisions and the acts of public officials, but it’s rare that a media outlet will understand Wikipedia well enough to track a contentious discussion effectively. Schow is ahead of the curve: she knows enough about Wikipedia to find some tantalizing tidbits, to generate copy, to generate clicks, and to influence those readers who lack deep familiarity with Wikipedia.

Specific problems with Schow’s account

But this story, despite its ramifications for an Academy Award shortlisted film and the National Football League’s #1 draft pick, was never picked up in any depth by a journalist who understands Wikipedia’s inner workings. Commentator Mary Wald did briefly note that Alva had observed Wikipedia standards, in a Huffington Post piece that highlighted the strength and stature of the interests taken on by the film. But this brief mention in a single story did not turn the tide. Anyone who follows the media coverage would likely be left with the incorrect impression that Chain Camera had done something wrong — to this day.

If a journalist had covered the story in depth, paying close attention to Wikipedia’s policies, norms, and best practices, they would have noted several flaws in Schow’s analysis. For instance:

1. Schow began with a common — and erroneous — premise: she assumed Wikipedia’s COI guideline, which recommends against editing an article while in a COI, is a policy. Wikipedia makes a distinction between the two, and explicitly notes that guidelines “are best treated with common sense, and occasional exceptions may apply.” Guidelines are not to be treated as rigid requirements. The COI guideline, in particular, has been scrutinized and deliberated extensively over the last decade. Wikipedia’s need for experts and its philosophical commitment to open editing have both prevented it from ever adopting a formal policy prohibiting editing while under a COI. Wikipedia’s relevant policy does not prohibit someone like Alva from making edits, but it does require disclosure in one of three places. Alva made that disclosure from the start, and in fact exceeded the policy’s requirement by disclosing in multiple places.
2. In her second column on the topic, Schow attaches significance to Jimmy Wales’ important-sounding words about changing Wikipedia policy in light of Schow’s report. This, again, is an understandable mistake; with most organizations, it’s safe to assume that a founder and board member’s ambitions have a close connection with reality. But with this particular board member and this particular issue, that assumption couldn’t be much further from the truth. Jimmy Wales has a long history of strongly advocating the “bright line rule,” which — had Wales’ efforts to have it codified a policy not been rejected — would have forbidden certain COI edits. Wales has even unequivocally stated that it doesn’t matter if the public thinks it’s policy; in his view, such details are unimportant. To put it simply, Wales is an entirely unreliable source on the topic of conflict of interest on Wikipedia. And despite Wales’ “renewed interest,” as Schow called it, his commentary on the topic ended as soon as it became clear the facts did not support his initial reaction to Schow’s column.
3. Schow doubled down on some of her strongest words about Alva’s approach, in her third column (November 30): she claimed that Alva had failed to sufficiently disclose his editing of topics related to The Hunting Ground until September 2015. But he had in fact exceeded Wikipedia’s disclosure requirements, as mentioned above. As she did acknowledge, Alva disclosed his connection to The Hunting Ground as early as March 2015, prior to any edits to related Wikipedia articles. He made further, more specific disclosures on April 23, July 27, August 10, and again on August 10, all before the September edit noted by Schow. A columnist, of course, might not be expected to fully grasp the intricacies of Wikipedia editing; but to vet such strong opinions before doubling down, she might have interviewed an uninvolved Wikipedia editor or two.

Schow’s errors may well have resulted from a good faith effort; but that doesn’t make them any less important. Her influence on the public perception of the connection between Chain Camera and Wikipedia has been substantial (see coverage at the Independent Journal Review and the Hill). So it’s significant that she got major parts of the story wrong.

Let the Wikipedia process work — don’t try to shut it down

In covering any story that challenges powerful institutions, Wikipedia editors have to sort through strong messages from various parties. Ultimately, Wikipedia relies on the sources it cites as references. High-quality source materials, not the interests or organizational affiliations of Wikipedia editors, should be the main factor in crafting its content. Wikipedians should not ignore those affiliations, and should always be mindful of the COI of various parties―not only of the editors, but of the people and institutions who generate and influence the stories they cite.

Any COI can be either disclosed or obscured, and even a fully disclosed COI can be managed well or poorly. Of course, it’s impossible to know whether other, anonymous editors have undisclosed COIs; but it would be foolish to conclude with any certainty that those who disclose are the only Wikipedians with a COI, when more than a decade of experience tells us that secretive paid editing — despite being a policy violation — is commonplace. Wikipedians should applaud Alva and Chain Camera Pictures for disclosing from the start. Even if they disagree with his specific suggestions or edits, they result from a good faith effort to improve the encyclopedia. When Wikipedians disagree with a good faith editor, they should talk it through — not discuss whether to block them from editing.

Wikipedia needs more, not fewer, expert contributors

When experts engage openly with Wikipedia, seeking to improve the encyclopedia, we should celebrate and support that effort. Chain Camera Pictures brought something to the table that few Wiki Strategies clients do: they sought to improve Wikipedia’s coverage of a broad topic they knew well through their work. Does this mean that they alone should determine the content of relevant Wikipedia articles? Of course not — Wikipedia’s model demands that any Wikipedian present convincing arguments, with reference to independent reliable sources. That is exactly what Alva did.

The approach Alva took, overall, is the right one. It should be readily apparent to any Wikipedian who looks at the edit history that Alva’s overall intent was to be transparent about his affiliation. Alva made several disclosures, and engaged other Wikipedians in discussion on points of contention multiple times. He added independent, reliable sources, sorting out disambiguation pages, reverting vandalism, and expanding content, and removed poorly-sourced, inaccurate information.

For a topic as important as sexual assault allegations on university campuses, Wikipedia benefits when experts engage with its content. Every day, non-expert writers do their best to build snippets of information into a narrative, to build Wikipedia articles; but it often takes some expertise to evaluate and refine that narrative. Wikipedians recognize this need; there is even a banner placed on articles deemed to lack an expert’s perspective.

This banner is placed on a Wikipedia article when somebody thinks an expert opinion could help. (See banner here)

The creators of The Hunting Ground are not, of course, the only experts on this topic. Investigative reporters like Walt Bogdanich of the New York Times and Kevin Vaughan of Fox Sports reported extensively on the subject. They reviewed thousands of pages of documents and interviewed many and various parties. In so doing, they surely developed significant expertise. If Wikipedia seeks to excel at summarizing all human knowledge, it should engage people like investigative filmmakers and journalists, who often have the strongest understanding of a given topic. As readers and as Wikipedia editors, we rely on these people to report on difficult stories that institutions often try to keep secret. We should applaud and welcome experts of all stripes when they bring their skills and knowledge to Wikipedia, as long as they are upfront about relevant affiliations. If it keeps the focus on including experts and sorting through disagreements, Wikipedia will be a more robust and comprehensive platform. Its editors, and more importantly its readers, will benefit.

Disclaimer

I own the business Wiki Strategies, and Chain Camera Pictures is a Wiki Strategies client. Our statement of ethics addresses cases like this; specifically, see item #4 under “Broad commitments & principles,” and the second paragraph of “Article composition and publishing.” It is unusual for us to blog about a client’s project, as we do here; in this case, the client made the decision (in consultation with us) to disclose our work together. In this blog post, we focus on the process Chain Camera Pictures followed in editing Wikipedia; in light of the issues addressed in our statement of ethics, we do not comment on the specific content of the Wikipedia articles in question.

Originally published at wikistrategies.net on November 1, 2016.