Five Questions with James T. Hamilton
What is the value to society of investigative journalism? In his new book, Democracy’s Detectives: The Economics of Investigative Journalism (Harvard, 2016), author James T. (Jay) Hamilton calculates the long-term economic consequence to society when lives are saved and disasters are averted by such reporting. He argues that citizens who live in a community are the real beneficiaries of investigative journalism, regardless of whether they are subscribers to or viewers of the media outlets that invest significant time and money in a reporting effort that surfaces a societal problems and analyze the causes. In contrast, most news organizations gain very little direct economic benefit. With revenues and profit under pressure in many legacy news organizations, fewer such reports are being produced. This “market failure” has long-term implications for society, argues Hamilton, which is why he hopes the book will be read by both journalists and non-journalists.
Hamilton, the Hearst Professor of Communication and Director of the Journalism Program at Stanford University, has a PhD in economics from Harvard University. His research on computational journalism focuses on how the costs of story discovery can be lowered through better use of data and algorithms. His previous books on media economics include All the News That’s Fit to Sell: How the Market Transforms Information into News (Princeton, 2004) and Channeling Violence: The Economic Market for Violent Television Programming (Princeton, 1998).
Click here to learn more about the book’s key findings. You can purchase the Democracy’s Detectives here.
How did you become interested in this topic?
While I was working in 2011 as part of the Federal Communications Commission Working Group on Information Needs of Communities, I was struck by the number of people outside of the agency who believed that the tumult in news markets was simply “creative destruction.” They believed that the public interest was defined by the public’s interest, that if a story was important it would be told, and that the decline of the media heralded a decline in the era of gatekeeping.
I take a different view. I think that public affairs stories involve what economists call positive externalities or positive spillovers. When a media outlet invests significant resources into discovering and telling a story that holds an institution accountable, it can be hard for the media outlet to capture the value for society that the story generates. The benefits may flow to many people who are not subscribers, and this makes it hard to generate sufficient revenue to fund these types of stories. The dearth of local public affairs reporting is probably the biggest market failure associated with news markets, but in an era of great variety and choice in many different types of information this gap in the market for public affairs stories gets neglected. I decided to write about the market for investigative reporting to see if I could document and describe the problems with the market’s under provision of this type of hard news. The process of researching and writing took five years, and the result is my new book, Democracy’s Detectives: The Economics of Investigative Journalism. In the book I offer case studies that show that each dollar invested by a news outlet in an investigative story can generate hundreds of dollars in net benefits to society from changes in public policies.
There has been a dramatic decline in recent years of newspaper revenues and profits, as well as newsroom staffing. How might this have an impact on the role of investigative journalism?
The decline in resources and staffing has likely translated into a reduction in investigative work at many local outlets. In looking at top investigative reporting awards, I found that prize worthy work was becoming more concentrated. The top five outlets accounted for 30% of major awards in the 1990s, a figure that increased to 47% in the 2000s. Prize winning work now involves more journalists per project and more partnerships across media outlets, with major projects becoming team efforts whose results can be spread across many different audiences. I found that the average age of reporters winning Pulitzers for investigative work has increased by nearly ten years since the 1980s, which raises the question of where the next generation of accountability reporters will gain experience and training. Public records play a central role in investigative work, but I found at a sample of federal agencies between 2005 and 2010 that FOIA requests from media outlets had declined by 25%. FOIAs by local newspapers dropped by nearly 50%, while those from other media such as Bloomberg, AP, and niche outlets aimed at those with a business interest in government increased by 42%. This shift in FOIA activity away from local newspapers is consistent with the Pew Research Center’s finding that journalists from niche outlets now outnumber those from daily newspapers in the US Senate Press Gallery.
Why should newspaper publishers want to fund investigative journalism? Did you find any financial benefit to a newspaper from pursuing investigative reporting?
I found that newspapers that did investigative reporting did have distinct characteristics: they had larger circulations, which let them spread the costs of investigations across many readers; were in market areas with a higher median household income, which can translate into better ad support; and often were associated with family ownership or dual stock structures which allowed individuals who cared about the public service role of the paper to direct more resources to this reporting. Investigative reporting can be part of a strategy of product differentiation. In an era where people have many contending outlets which can offer the breaking news of the day, distinct stories allow you to stand out. If you are in a competitive news market online, price gets competed down to the marginal cost of offering one more page view, which is zero. You are in a competitive market if you are offering commodity facts that many people are repeating. Product differentiation comes when you tell a different story that does not have close substitutes. That story might be distinct because you’ve uncovered new facts, or because you’re telling the story in a particularly effective narrative, or you’re telling the story through people distinctive for their experience and voice. The long-term brand created by the expectation of new, important work is hard to measure. But in my analysis I did find companies such as McClatchy, which in the 2000s submitted at high rates to prize competitions run by Investigative Reporters and Editors, in the 1990s had participated in Civic Journalism programs, and in the 1970s and 1980s had CEOs who talked relatively more about public service than profits.
Tom Rosenstiel at the American Press Institute has done great work showing how at newspapers, “Major enterprise stories scored 48 percent better than others in a measure of overall engagement.” Longer stories do generate higher traffic and engagement. But even though investigative work can draw higher engagement, from the perspective of economics newspapers will still not be able to capture the full value generated when policies change because of the reporting. That means it will be underprovided in the market, and that sets up the challenge of thinking about how to better support it.
Most of the papers in this country are small, with a circulation of less than 15,000. What advice do you have for small community newspapers?
In my analysis of investigative work submitted to IRE prize competitions, I found that smaller newspapers devoted a greater share of their stories to very local topics such as education or community development and housing. These are the type of stories where they have a comparative advantage — they are much closer to these local policies, and these stories can differentiate them from news generated by outlets with a wider geographic audience and hence a wider geographic focus in stories. These stories can generate results that are individualistic (e.g., firings, resignations), deliberative (e.g., hearings or debates), and substantive (e.g., new policies). I found that 11% of the IRE submissions by smaller newspapers, for example, generated further investigations by others such as government agencies.
Advice for community newspapers would include:
- Seek subsidies. IRE has offered training subsidies for small newsrooms and journalists from small newspapers. The Fund for Investigative Journalism offers grants for specific reporting projects.
- Seek allies. ProPublica will often partner with local media outlets as a way to get more eyes on the ground on stories that have a highly localized dimension. ProPublica also publishes databases and reporting recipes that allow smaller newspapers to create a localized version of a broader story. Other nonprofit media outlets such as Reveal can be potential allies to join forces with on a story. The Google News Lab exists in part to help journalists discover local stories from Google data, and their tools and data on search can be places to start in the quest for new stories.
- Seek new data. Right now at the state and local level data are being released online that can be the source of new accountability stories.
What lessons might non-journalists take from your book?
The central message of the book is relevant to all interested in how our democracy is functioning, namely that investigative journalism is underprovided in the market but new combinations of data and algorithms can make it easier for reporters to discover and tell the stories that hold institutions accountable.
Two of the main lessons from the book are hopeful. One is that if government officials came to acknowledge the market failures associated with investigative reporting, then their actions could translate into policies that would reduce the hassle costs of investigative work. This could include real implementation of the newly passed FOIA reforms, adding journalism as a field to support in federal R & D competitions focused on algorithms, tech, and data, and better implementation of open government and transparency policies.
A second lesson is that the evolving field of computational journalism holds the promise of lowering the costs of discovering investigative stories and of telling stories in more personalized and engaging ways. The field will advance more readily if:
- Computer scientists who value the social gains from accountability take on research challenges in story discovery, telling, and distribution
- Philanthropists focus on research questions about truly personalized news, accountability by algorithm, transcription and text mining, and analysis of corporate and government data for story discovery
- Journalism educators work across disciplines to draw researchers to journalism challenges, and fundraise to support journalism scholarships for those headed for public affairs reporting
- Reporters try news methods for social impact through new tools in discovery and storytelling
The actions of these different groups in combining computation and journalism may offer an expanded set of people new ways to hold power accountable, and allow them to serve as democracy’s detectives.
Originally published at Center for Innovation & Sustainability in Local Media.
