Can you really benchmark experience?

Quantifying the qualitative is difficult, but could be useful.


I spent the last several days in Austin, TX for the annual South By Southwest conference. As someone whose job revolves around social media, using a computer and communicating online, it’s refreshing to get out and engage with people directly. Attending sessions, going to parties and meetings, and conversing face to face rather than via Google Hangouts has a variety of benefits.

Inevitably, upon ones return, the question frequently asked is, ‘how was it?’ The easy thing to do is say, ‘it was good!’ and nod enthusiastically, sharing an anecdote or two. Rarely do you really stop to analyze the experience on a deeper level. It was good? How good? Better than previous events? How much better? Better in what way? It quickly becomes clear that benchmarking an event can be a difficult proposition.


The reason for this of course is because experiences are linked to emotions. An experience is as much about how you felt as what you did. Of course we can have good experiences and bad experiences and rank them accordingly, but most of life’s experiences are much closer together on the continuum. It’s a matter of degrees, not absolutes.

Of course you can try to benchmark things like SxSW by putting a veneer of rigor against it. Prior to the event, Sprinklr published an ROI calculator to help you determine if your time in Austin was well spent. A worthy effort I suppose, but always one doomed to be superficial at best. It’s hard to turn an event like South By into something purely quantitative.

The idea of exploring emotions and social media is not new. You’ll recall that last year their was a controversy as it was discovered that Facebook conducted a massive psychological experiment on 689,003 users, manipulating their news feeds to assess the effects on their emotions (Details here).

What I have in mind is something a little different and more directly connected to content published by brands, since that is our focus at Unmetric. Let me explain by way of example. It’s the Super Bowl and your favorite team is playing. You’re watching at home with friends, wearing a team jersey, eating your favorite foods and enjoying the game. Ultimately your team is victorious and Brand X, a Super Bowl sponsor, posts a status update on Facebook congratulating your team on their victory. You enthusiastically give that post a thumbs up.

Three weeks later, Brand X posts another status update, this time wishing all its fans a ‘Happy Wednesday!’ Sure, why not give it a like.

Is it the same action? Yes, but are all likes created equal? In other words, does context have to be figured into the equation? Does Facebook need to create some sort of sliding scale? Perhaps fans could give posts a score ranging from one to five ‘thumbs up’ to give marketers a truer understanding of their feelings.

Currently social media platforms are relatively simplistic in the options they provide for consumers to express the level of emotion they feel with actions like favoriting, retweeting or liking and so on. It’s purely quantitative. By adding another, more qualitative dimension they could provide marketers with a deeper understanding of how context affects the true opinions and emotions of consumers in a quantifiable way.

Algorithms are good for many things, most significantly handling things at scale. But nuance and emotion are still better handled by humans. At Unmetric, we’ve married human analysts and algorithms and attempt to do for benchmarking what CAPTCHA did for AI. Does it work? We’re still finding out.

Like what you read? Give Rick Liebling a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.