Medium is Curating a Public Debate — What Criteria Do We Use to Evaluate its Success?

Getty Images

Starting today, and for the next two weeks, Medium is curating a conversation about the criminal justice system — specifically mandatory minimum sentencing and the war on drugs, though all related topics are fair game.

This is important and interesting for any number of reasons, not least of which is the potential to spark a highly visible discussion among advocates and adversaries on a topic that impacts so many lives. In the most utopian of terms, it also offers up a new possibility for reasoned debate and civilized conversation in a political system currently incapable of either (indeed, this is the theoretical promise of Medium).

But how will we know if it succeeds? What does success for something like this even mean? Success for who? Success at what?

As Medium kicks off this experiment, here are a few thoughts on varying levels of criteria that could be used to judge success — both narrowly for Medium as a platform, but also more broadly for advancing the state of democratic dialogue and debate.

Impact

Let’s start out by going big. Medium has set the bar high here by purposefully choosing an issue that is both top of mind publicly and on which there is active legislation in play (the Sentencing Reform and Corrections Act). In that context, what are the potential impacts of the project?

• Creates Legislative Change: While it is not realistic to attribute passage of legislation to any one effort, it is within the realm of possibility for small tweaks to the bill to emerge (i.e. amendments and language adjustments that arise from persuasive arguments) that could be attributable to dialogue on Medium. This would be the ultimate measure of success, though probably a metric that is out of reach, and certainly one that raises thorny issues for Medium as a presumably neutral platform.
• Creates Public Perception/Awareness Change: Can one or more pieces published in this debate (or the two-day “Town Hall” capping the debate) spark significant press coverage or go viral in some corners of the internet in a way that gets people reconsidering their opinions? If Medium can be a conversation starter (or mover) in that way for ideas outside of the Tech/Media/Design/Marketing sphere that currently dominate its pages, it would increase the attractiveness of the platform for future matters of public debate and validate Medium’s role as a new kind of public square/bully pulpit.
• Builds New/Strange Partnerships: Through engagement in the debate and direct conversation with people outside their particular sphere of activity, do new partnerships arise that offer the possibility for changing the playing field?

Any one of these would be a solid win — for Medium or for democratic debate— though it should be noted that these are medium- to long-term impacts and they will be hard to judge as the official debate closes.

Quality of the Debate and Improvement of the Public Sphere

If the metrics above are moonshots — the results that Medium and observers of public discourse would love to see — the criteria for success laid out below are very much within the power of Medium to accomplish, and are solid indicators that Medium could eventually achieve some of the higher order impacts listed above.

• The Debate Pops the Filter Bubble: Can Medium as a platform surface both sides of a debate to users? Unlike Twitter or Facebook, which create and foster filter bubbles based on who you follow, your likes and dislikes, Medium places just as much importance on the topics you follow, making no judgement (to my knowledge) about your relative ideological position on the issue. The possibility for Medium to pop filter bubbles exists, the question is how robust it is and how much is automatic vs. how much do users need to seek out that experience? This is a subjective experience for each user, and a hard one to track. But certainly one I’ll be looking at anecdotally as I follow the conversation.
• Delivers an Ideological Diversity of Participants: From a political perspective (since we are considering legislation as a potential outcome), is there an ideological diversity of opinions represented in the debate? Are people whose votes can truly be moved participating in the debate, or are participants just party figureheads on the platform to spout a party line?
• Delivers an Experiential Diversity of Participants: Politics aside, is Medium bringing to the table all the different perspectives that could inform this debate, and is it giving them each enough prominence to break through and be in tension with each other? This could include everyone from non-violent drug offenders to beat cops, families of victims to police superintendents, municipal officials and prison administrators to defense advocates.
• Engagement Between Stakeholders: This is the difference between Medium offering a soapbox and Medium hosting a conversation. Do stakeholders representing conflicting viewpoints respond directly to each other’s posts, or are they shouting into the void and only engaging with like-minded individuals? Are they responding at all? Are we seeing truly threaded discussions (like this discussion on Media and Tech), or do varying viewpoints quickly move into silos?
• Public Shifts in Individual Opinion: Through persuasive argument and engagement, does one or more prominent stakeholder in the conversation change their their position on a major or minor piece of the issue or the legislation? That kind of personal opinion shift — made publicly — would validate participation on Medium as a true venue for persuasion, rather than another echo chamber for each side to leverage for its own partisan ends.
• Demonstrates Sustained Interest: For Medium in particular, this project is part of a bid to become the platform of choice for thought leaders and stakeholders seeking to mold public opinion. Medium is putting a lot of muscle into ensuring that those key players are on the platform for the next two weeks. The question is, do they stay? For this to be truly successful, the conversation needs to continue, and hopefully grow, absent the guiding hand of Medium staff.

Successfully executing many of these elements is squarely within Medium’s control, and their announcement already showed progress — at least on delivering a diversity of participants to the conversation. If Medium can score high marks on these criteria they’ll be setting themselves up for larger success. If they fall down on any one of these, it will be hard to see how they can eventually achieve the impact goals I’ve described.

Grading the Debate

My strong suspicion is that while Medium’s experiment will excel at the second set of metrics/criteria (improving the debate/quality of discussion), we won’t know about larger impact until some point in the future (and maybe not until Medium repeats this experiment 2–3 more times).

The biggest wildcard in all of this is the live “Town Hall” hosted by Medium at the end of the two week conversation. One of the big advantages of Medium is the fundamentally asynchronous nature of the conversation. A live Town Hall breaks that asynchronous nature, but maybe to good effect.

We’ll see.

In the meantime, check out this post by elizabeth tobey to get the full rundown on what Medium is trying to do, why, and how you can participate.

Let’s talk about the criminal justice system

To get updates how Medium can continue to improve its platform in service of public discussion and debate, follow Design the Debate by clicking “Follow” below.