By the Numbers- Looking Back on the GIN Project’s First Year
The GIN Project is an effort to understand the ecosystem of global ratings and rankings. The ratings and rankings, or indices, inform decisions in business and policy. We’re interested in not only how power manifests itself in the act of scoring such intangibles as ‘freedom’ and ‘prosperity’, but also how scoring strategies themselves proliferate in the tangled network of publishers.
Our project is inherently complex. Before we considered how indices might borrow from one another, the task of deconstructing these behemoths of abstraction occupied weeks of our time. Even for indices within the same thematic grouping, we found scoring strategies to be wildly diverse. In order to deconstruct our initial set of indices, we had to develop a coding scheme that could account for all of the different ways of defining, rating, and ranking issues of global interest. No small task. In fact, we continually found ourselves needing to interrogate our assumptions, reevaluate our definitions, and update our codebook.
With coding finally underway, we turned our attention to how the indices might all connect. We developed Python scripts to compare the spreadsheets generated for each index. Later, additional scripts were developed to translate from our coding scheme to the strict format network analysis requires. We celebrated our initial results. We identified errors in the tangled network maps we drew. We adjusted our approach. We coded some more. We began to see the network take shape.
It’s difficult to estimate how many hours our dedicated team of coders spent deconstructing our initial set of indices. Several indices were found to be well-documented, straightforwardly constructed, and conservatively sourced. Other indices proved to be unwieldy monsters, with minimal documentation, and hundreds of lines of subindices and sources.
Upon generating the network from the deconstructed indices, we were faced with a beautiful illustration of the complexity we had corralled: the diverse scoring strategies of dozens of indices rendered into comparable constellations. We also possessed a representation of all the work we had put into the project.
We’re very proud of this work. All of these results, however, we consider to be only this project’s very first steps toward a larger goal: understanding the complex dynamics at the heart of the global ecosystem of ratings and rankings.
