Those reports on Google funding academic research may be 99% nonsense — but we should talk about that 1%
With thanks to Molly Sauter for reporting help
A large Silicon Valley corporation is funding research that serves its interests. But it’s not the one you think.
That research in question comes from the opaque Campaign for Accountability and its Google Transparency Project, whose only known funder is Google rival Oracle Corporation, claiming that research issued by dozens of independent scholars is bought and paid for by Google.
First, a recap: Last Wednesday, both the Wall Street Journal and the Chronicle of Higher Education published articles repeating and amplifying the claims of a new “report” (a list, really) published by the Google Transparency Project at the Campaign for Accountability. Before long, academic Twitter was abuzz: stories about ethical improprieties and shoddy research methods draw PhDs like moths to a flame.
Drawn in by the Chronicle’s salacious headline, which repeated the WSJ’s barnburning claims, I reviewed the Google Transparency Project’s list, noting again and again the names of scholars I knew to have no Google connection whatsoever, or to have only the most tangential tie. Many of my colleagues (perhaps even you!) were surprised to find their own names on the list, owing to their past Google Policy Fellowships (or their co-authors’) or their affiliation with research centers that themselves were indeed funded by Google. A good number expressed their bewilderment on social media (as you do).
A number of academics (many of them active on Twitter) were subsequently contacted by the Chronicle of Higher Education, which published a follow-up story last Thursday. As the Chronicle laid out, the GTP database is full of names of researchers who either have never received money from Google, only had ties to groups that themselves had some connection to Google, or had previously received a Google Policy Fellowship to spend a summer working at a non-profit. Among the more bewildering cases was PhD candidate James Losey: the report claimed that a paper he published in 2011 was funded by a fellowship he received in 2014 (GTP acknowledged the error, but kept the paper in the database because Losey’s co-author had been a Google Policy Fellow in 2008).
Confused scholars turned to the report’s methodology for clarification. As best I can tell, the process went something like this:
- First, the GTP collected the names of individual researchers, as well as the names of think tanks and university research centers, listed on Google’s corporate site as having received funding from the company.
- Next, they determined which of the industry groups to which Google belongs (such as the Internet Association) give out research grants, and identified the recipients of those grants as well as individual scholars associated with funded research centers — what the report calls “indirect support”.
- Having cast the widest net possible, the GTP then queried Google Scholar (ironically enough) for papers authored by those scholars (or scholars working at research centers vaguely associated with Google) that included certain policy keywords. All papers published after the first “Google grant” were deemed to be supported by Google.
- The GTP then calculated how many of those papers mentioned Google in the acknowledgments section. Those that did not (many for the good reason that Google didn’t actually fund the research) were classified as failing to disclose support, the implication being that they had secretly been bought and paid for by Google.
For an initial back of the envelope calculation, that’s actually not that bad of a first step. Casting a wide net to collect the entire broad universe of possible cases before narrowing it down is valid — but you have to actually narrow it down. At a minimum, GTP should have excluded the former Google Policy Fellows and the scholars whose only Google connection was an unpaid connection to a center that itself received Google funding. I also don’t buy the claim that funding for a specific project taints a scholar’s work for all eternity, for reasons that Casey Fiesler spells out. Additionally, the researchers failed to demonstrate a causal relationship between Google funding and the substantive content of the papers in the database. In other words, correlation isn’t causation, though it’s a good reason to look into the possibility of causation. While “follow the money” is a good, and important, mantra for journalists and watchdog groups, financial and institutional ties can only suggest the existence of influence or a quid pro quo. A much deeper analysis is needed to prove if and how funding actually impacted the research findings. That’s where GTP’s claims don’t add up.
Casting a wide net to collect the entire broad universe of possible cases before narrowing it down is not a bad first step — but you have to actually narrow it down and identify a plausible causal mechanism.
Many of the scholars and individual papers on the list are in fact highly critical of industry in general and internet companies in particular. For all its references to Google’s “public policy interests,” GTP fails to draw a meaningful link between those interests and the research allegedly supporting them (spelling out what those interests are would have been a useful step). Each paper in their database is tagged as pertaining to a subject area of some relevance to Google’s activities, all of which are extremely broad (such as Advertising, Antitrust, Copyright, First Amendment, Net Neutrality, Patents, Privacy, and “Other”).
Beyond the links by association that their list creates, there is no evidence offered that GTP confirmed that the papers actually supported Google’s positions or interests. GTP doesn’t even seem to define those positions or interests. Messy criteria for inclusion in this list yields a result that is misleading and error-riddled, but established media outlets are repeating its alleged outcomes with click-bait headlines that are all most of the public will ever see.
After an outpouring of criticism for their methods from academics and policy analysts, GTP published an addendum on Friday, in which it acknowledged the critiques of its methodology and findings — largely culled from scholars’ Twitter feeds — but insisted that their broad approach was an adequate and appropriate mechanism to demonstrate financial ties between Google and researchers who “support its policy views.” I fundamentally disagree with this premise, but but by that logic, is the Google Transparency Project itself hopelessly beholden to its own funders?
So let’s talk a bit about the Google Transparency Project. Its parent organization, the Campaign for Accountability, has only existed for about two years (lengthy Wikipedia entry notwithstanding), and therefore hasn’t had to file IRS Form 990 yet, the main way that non-profits disclose their funding and other financial records to the government and to the public. In fact, the Campaign’s only known funder is Oracle, which has been waging a well-documented legal and media relations battle against Google for years. The Campaign for Accountability’s executive director, Dan Stevens, told Wired that they don’t disclose their funders and want to “let the work speak for itself” — odd double standard, isn’t it?
There is no telling who else might be funding the group, but in the absence of evidence to the contrary, it is reasonable to assume that the Campaign for Transparency’s research advances Oracle’s public policy interests (again, using their own logic).
In addition to the Google Transparency Project, the Campaign for Accountability also runs a similarly-opaque Investigation into the Solar Industry, as well as projects on more generic topics like Federal Accountability, State Oversight, and Corporate Accountability. A cursory review of the group’s staff and board members reveals that the organization’s small staff lacks anyone with advanced research training or expertise.
It also criticizes scholars for allegedly failing to do things that it doesn’t do itself, like disclosing its funding sources and subjecting work to a rigorous peer-review process.
This bears repeating: The Campaign for Accountability’s Google Transparency Project selectively targets individual researchers and think tanks/research centers on the basis of their research grants — while refusing to disclose its own funding. It is more than a little strange that a group that claims to be devoted to “exposing misconduct and malfeasance in public life” hides its own funding sources. The idea that said mission is best accomplished by listing decontextualized funding streams to research and researchers is a flawed and confusing premise, but even by that measure, the Campaign for Accountability fails to live up to its own standards.
Again, the Campaign for Accountability’s only known funder is Oracle, and as I tweeted earlier this week, academics are just collateral damage in the Oracle/Google wars. I don’t think it’s a coincidence that the GTP report and the first set of stories based on it went up just in time for the Net Neutrality Day of Action. Support for net neutrality is one of the few major policy differences between Google and many other ICT companies.
I strongly suspect that part of the goal here is to undermine policy research and advocacy that support net neutrality. Tech journalists: that is what you should look into.
The conversation we should be having
My point here is not to defend Google or its role in the information and research ecosystem (the company did so itself on its Google Policy blog). There is much to be criticized there, as many scholars on the CTP list have done and continue to do. As the Electronic Frontier Foundation’s Jillian C. York tweeted, “Google absolutely tries to influence NGOs and academics. But that doesn’t make CfA’s work magically good.”
As is so often the case, there is a tiny kernel of truth to the story: Google — and other companies — have tremendous amounts of influence in policy, research, and public life. This should be investigated and debated in a rigorous, seriously-minded way. Sloppy research like this only muddies the waters and makes it harder to have serious conversations about the real problems, as Fortune pointed out last year. This list, and the subsequent journalistic takes that have validated it, have the strong potential to taint the public and policymakers’ view of research as inherently biased and no more valid than unsubstantiated opinions — this may even be part of the goal.
As computer science professor Casey Fiesler wrote on Medium, the GTP report “plays into dominant narratives that tech companies are evil. But it also plays into an increasingly troubling narrative that science can’t be trusted. In an attempt to discredit Google, this project’s report also implies that academics can be easily bought off. That research is for sale to the highest bidder. It is an easy leap from this concept to thinking that a political party is paying for climate change research, for example — and this is a narrative that is dangerous right now.”
This line of thinking also creates new problems that are potentially just as dangerous as Google’s influence on public life. According to the more extreme voices among the funding transparency ideologues, all work is tainted by its funding sources, and influence persists as funds are granted, re-granted and sub-granted — a standard practice in the non-profit and academic research world. Academics address these concerns using common practices, such as acknowledging direct funding of research in research papers themselves, or recording such awards on their curricula vitae. This is also why we painstakingly describe our research methodologies and subject our work to often torturous peer-review processes (cheers to you, Reviewer 2).
If the funding purity extremists had their way, research funding of any sort would render work illegitimate, and such funding would be discontinued altogether. The implications of this affect people well beyond the confines of the university campus: growth and innovation in technological, medical, engineering and many other sectors comes directly from university-based research and development. Much of that transfer to industry is what directly spurs economic growth, production and jobs. Yet universities, and the professors who work in them, are increasingly under fire by critics who take issue with everything from campus policies to research outcomes. The politics of this position cannot be avoided, especially as the Trump administration threatens to decimate federal research funding.
Moreover, many people would argue that big tech companies have a responsibility to fund research, non-profit groups, and conferences as a way to give back to society. The tax code incentivizes such gifts for just that reason. There is a right way and a wrong way to go about such funding, well worthy of a conversation, but that is not the GTP’s argument — rather, it is simply to suggest taint by association.
Strangely, the “taking funding from X is always bad” crowd never seems to have alternative suggestions for how research and other public-interest work should be funded.
Finally, the GTP’s framing of the issue, which effectively implies that anyone remotely associated with Google is intellectually corrupt, makes it all but impossible for many of the people who know the topic best to join the conversation. The scholars whose activities were inaccurately described in the GTP report risk accusations of self-serving bias if they speak up — caught up, unwittingly and unpleasantly, in the GTP’s attempt to score anti-Google points. Indeed, the GTP’s definition of “Google ties” is so overly broad that it forces academics, researchers, policy specialists and journalists to consider the appearance of overblown conflicts of interest that do not actually exist. No one wants to be falsely portrayed as standing up for Google or being against transparency.
I hesitated to write this piece for precisely this reason: I have held a handful of fellowships at New America since 2014, all of which were funded by foundations or government grants (my current fellowship ends this month). New America, in turn, receives some funding from Google for core operations and some of its program work. I have a dissertation to write. I don’t really have time to deal with Internet trolls calling me a Google shill.
But I decided to push through the chilling effect because this is actually an important conversation to have. Private companies in general, and tech companies in particular, play an incredibly powerful role in society. We can’t rely on governments to adequately regulate them, especially not now, especially not in the United States. Not only is the federal government in shambles, technology and the political economy of the Internet are evolving too fast for us to properly grapple with. Trained, experienced researchers inside and outside of the academy are among the best positioned to engage with the thorny question of how tech companies should be studied, using what methods, and what conclusions specific studies can and can’t support.
