The news ecosystem mapping quandary: Balancing depth and scale
To date, most research has sacrificed one for the other; that’s why we are testing a new method to do both
This is the first post in a series about the Center for Cooperative Media’s efforts toward deep and replicable mapping of U.S. local news ecosystems, with an emphasis first on testing in New Jersey.
It’s a series of questions we get often at the Center for Cooperative Media: How many news organizations operate in New Jersey? How many are print versus radio versus television? How many people do they employ? So what parts of the state have no local news source? How can we help those places?
The problem is, we can’t answer these questions with complete accuracy. In fact, we don’t know anyone who can. Despite the volume of research currently under way about news ecosystems, there is no gold standard; many studies to date have critical flaws, such as focusing on only one type of media, using too few sources to feed underlying databases, or considering news only through a strict geographic lens.
We believe that knowing the true landscape of the local news ecosystem in this digital age could help our work, and others, in so many ways.
More than a year ago, the Center began work on a new method to address this urgent question.
We’ve been considering the work a three-phase process. Phase 1, now complete, was a literature review that summarized the entire ecosystem/ecology literature to-date, organized the studies according to eight different typologies, and fleshed out a new method that will allow for both depth and scale in the study of local news ecosystems. Phase 1 was co-authored by myself, Magda Konieczna, assistant professor of journalism at Temple University, and Jesse Holcomb, assistant professor of communication arts and sciences at Calvin College.
This series of blog posts will address different issues and aspects of news ecosystem mapping. In this first post I will discuss one aspect of the research addressed in Phase 1: The problem of depth versus scale. This is, in some ways, the fundamental issue with which all research must grapple, but it is especially acute in ecosystem mapping because of the amount of data needed to understand a local news ecosystem, and the desirability of a comparative element, which requires a larger sample of ecosystems.
Combining depth and scale requires either a nearly unlimited amount of resources or an incredibly sophisticated research design utilizing the latest digital tools. Because we do not have a nearly unlimited amount of resources, we are striving for a brilliant research design using the latest digital tools (more on the research design in a later post).
By depth I mean building out a comprehensive census of outlets, an accounting of news flow that includes key producers and amplifiers, and an assessment of corresponding social media. Depth may also, but need not always, include understanding consumption patterns and effects. By scale I mean producing this knowledge for many local news ecosystems, not just a few or a handful. Obviously, doing both is the gold standard for understanding a regional, state, or national local-journalism landscape.
Ecosystem studies that aim for depth are usually case studies. Case studies examine a specific place or event and use data collection tools that cannot be easily scaled, like on-the-ground explorations or analog studies of particular news stories flowing through the ecosystem. The scaled/scalable approach, on the other hand, employs methods capable of spanning geographies or news events: surveys, large datasets, or scraping software. Some such ecosystem studies are not technically scaled but are scalable.
Ecosystem case study methods may include fieldwork, ethnography or manual content analysis. The Pew Research Center (2010) study of news diffusion in Baltimore, for instance, reconstructed specific local news events by manually analyzing iterations of stories as they evolved in print, broadcast and online. The actors were enumerated through reportorial investigation — digital groundwork, email messages and phone calls to journalists and other stakeholders. Another good example is Anderson’s (2010) study of the Philadelphia news ecosystem, which came out of time spent in Philadelphia newsrooms and activist communities, combining network ethnography and qualitative newsroom analysis. These are classic case study designs. The amount of labor involved in replicating these works at a larger scale is outside the realm of possibility for any academic or industry study.
Pew Research Center (2015) also sought depth when it looked at the local news ecosystems of three American cities — Denver, Macon, and Sioux City — to understand how local news was transforming in the digital age. They conducted surveys of residents, content analyses, built outlet censuses, and performed social media analysis in each city. What resulted was a detailed analysis of the cities both individually and in comparison, from which they produced much insight. However, the methodology was so detailed and labor-intensive that it could never be replicated for a large number of cities.
Aiming for scale, Napoli, Weber, McCollough, and Wang (2018) sought to compare the quantity and quality of local news for 100 communities around the U.S. Using digital methods that included scraping more than 700 local news websites, they were able to both catalogue all outlets with an online presence in each community and study whether their output lived up to its democratic mission of providing original news about the community that fulfilled a critical information need. They included only outlets that were physically located within each community, leaving out any that might serve the community from elsewhere. This method produced important comparative results, but sacrificed some depth and nuance because it defined the local news ecosystem according to strict geographic parameters.
Another example of a scaled study of local news ecosystems is Abernathy’s (2018) study of the closure of local newspapers. As struggling locals are acquired by non-media corporations, Abernathy found, they are being closed or turned into “ghost” papers, which provide little original hard news about the communities they purport to serve. This study covers the entire United States at the county level — clearly a scaled effort — but because it only looks at newspapers, does not give us an accurate picture of the local journalism landscape in the digital age.
A number of published studies may be scalable, allowing proliferation of data while keeping research labor in check. These include network analyses, such as Graeff et al.’s (2014) study of the diffusion of news about the killing of Trayvon Martin, or Gordon and Johnson’s (2011, 2012) studies of the Chicago digital news ecosystem. Neither were scaled, but their leveraging of computational tools to collect data programmatically opens up broader possibilities.
As we can see, case studies offer more nuance and depth, but often sacrifice generalizability. Scalable studies can offer generalization, but often sacrifice detail and depth. The goal of the Center for Cooperative Media’s local news ecosystem mapping project is to devise a methodology that does both.
Sarah Stonbely is the director of research at the Center for Cooperative Media at Montclair State University. Contact her at stonbelys@montclair.edu.
About the Center for Cooperative Media: The Center is a grant-funded program of the School of Communication and Media at Montclair State University. The Center is supported with funding from the John S. and James L. Knight Foundation, the Geraldine R. Dodge Foundation, Democracy Fund, the New Jersey Local News Lab Fund of the Community Foundation of New Jersey and the Abrams Foundation. Its mission is to grow and strengthen local journalism, and in doing so serve New Jersey residents. For more information, visit CenterforCooperativeMedia.org.
