Sentiment Analysis: A better way to measure success
Engagement metrics and focus groups will only take you so far in understanding your audience
Congrats! Your social campaign got 500,000 impressions. That means it was a success, right? Well, not necessarily.
Were those positive impressions? Negative impressions? A mix of both? Did the campaign resonate with your audience the way you intended? Engagement metrics, like impressions, can provide a concrete definition of reach, but likely won’t provide a sense of what emotions or attitudes a campaign inspired.
Driven by the proliferation of two-way communication and feedback across social media sites, marketers are improving their understanding of audience perceptions. Through social sentiment analysis in particular, analysts aggregate and evaluate written reactions on social media to determine public attitudes about their organization’s activities and campaigns.
These reactions draw far beyond typical engagement metrics, such as likes or shares. Instead, sentiment analysis platforms interpret language in tweets, Facebook posts, blogs, Amazon reviews, and other online opinion forums. While aggregating the number of engagements can indicate the extent of a campaign’s exposure, sentiment analysis allows users to understand the essence of that exposure.
The Semantics of Sentiment
One form of sentiment analysis assesses semantic orientation. Data analysis platforms will sift through posts and code them as positive, negative, or neutral. This is helpful — to an extent — in determining whether a campaign was helpful, ineffective, or even counteractive.
But as content marketing and competition for social media engagement intensifies, marketers need to understand specific reactive emotions, not just positive or negative, good or bad. To further interpret negative attitudes, for example, there’s a major difference between disapproval and anger. More importantly, strategies to remedy those negative perceptions differ greatly.
Platforms to Peruse
A variety of evaluation and interpretation methods exist, but it ultimately comes down to choosing between a human-powered platform or a machine-powered platform.
The most effective human-powered platforms crowdsource people to complete tasks. Popular platforms, like CrowdFlower and Amazon’s Mechanical Turk, gather the critical mass required to efficiently process vast amounts of data, and allow users to upload data sets and instructions.
While originally a platform to convene a “crowd” to accomplish extensive tasks, CrowdFlower has been used so frequently to analyze social sentiment that it has begun training people to conduct sentiment analysis. Turkers, however, are largely untrained and receive only brief instructions.
Among crowdsourcing data analysis platforms, Mechanical Turk tends to have the lowest per-task cost. The tradeoff here is with quality control — other platforms have more robust quality assurance and qualification processes.
You’ll likely benefit the most from human-powered processes if you have smaller, project-based data sets. If you have time-sensitive deadlines, however, a machine-powered platform might serve you best. People will complete the tasks as they wish, so there’s no guarantee about how long your task will take.
While making your decision, remember that machine algorithms continue to refine their identification and categorization of human behavior. Innovations in natural language processing are helping some social sentiment platforms build a strong case for artificial intelligence over human intelligence.
For example, by using Crimson Hexagon’s NLP platform, Nespresso established and tracked reception of targeted marketing campaigns. Initially, Crimson Hexagon identified how and why people drink coffee (The result? It’s a social activity.) in order to identify new market opportunities for Nespresso, focusing on luxury appeal.
Crimson Hexagon used its sentiment analysis system again to gauge public reaction to these very marketing efforts, determining through traditional social media posts in addition to online travel reviews that in-room machines in hotels create successive and lasting positive buzz online. NLP also revealed an increase in public intent to purchase a Nespresso machine, rendering the social marketing campaigns successful.
Canvs, another social sentiment platform, boasts the ability to code for 56 different emotions in its analyses. Viacom has incorporated Canvs’s natural language processing capabilities as part of its proprietary Viacom Velocity product, which evaluates the impact of its partners’ marketing campaigns through social media feedback.
Advancements in natural language processing have allowed computers to overcome the nuances that have made millennials (and soon Generation Z) so difficult to understand. Sarcasm, intentional and/or careless misspellings, words with multiple meanings (e.g., sick) and ever-changing slang make keeping up with young people incredibly challenging.
Still, there are limits to human intonation that NLP at its current state cannot necessarily understand. While many platforms may claim to understand complex nuances of the contemporary English language, instances of sarcasm or statements with multiple meanings (e.g., You’re killing it!) could certainly slip through the cracks of imperfect artificial intelligence systems.
You should consider machine-powered natural language processing if you are operating at a large scale. Most NLP platforms discount their per-task cost as the data set increases. If you have a lot of diction data and need it analyzed in a timely manner, using a machine-powered platform is your best bet.
Accurate Analysis
Television networks (especially when targeting the coveted 18-to-34-year-old age group) frequently use Twitter as a critical source of feedback: The CW used Canvs to identify what emotions resonated most with viewers in an episode of Arrow. This sentiment analysis found the various emotions linked with each character and performer, identifying which characters attract the most powerful emotional interest, ultimately helping the network make more educated content decisions.
So, why pursue social sentiment analysis in market research efforts when you still have trusty, traditional research methods? Typically, market research answers questions about audience and content through focus groups or surveys. But because the feedback is shaped by the leading questions, the sentiment expressed isn’t necessarily raw or authentic, especially when limited to yes-or-no or multiple choice answers.
Social media data mining, on the other hand, allows researchers to identify what people are actually talking about less deliberately, and more spontaneously. Furthermore, instead of extrapolating from a small sample size, the ease of analyzing vast amounts of data often allows you to analyze an entire population, or close to it (e.g., female students on Twitter aged 18–22, enrolled in college).
More than Marketing
Applications of sentiment analysis reach far beyond marketing efforts. Financial analysts leverage sentiment analysis on Twitter and other social media sites to understand public thought toward companies in an ambitious effort to foresee global market behavior.
Bloomberg and Thomson Reuters have both incorporated Twitter feeds as principal components of their best-selling financial products. IBM and Intel have combed through responses and posts on their internal employee-facing websites to comprehend employee feedback for HR policies or developments.
And as public polling falls flat in predicting accurate results (remember the Brexit shock?), social sentiment analysis grows increasingly credible for elections and policy issues.
The rising popularity of sentiment analysis is based on its ability to quantify what has always been qualitative in nature. Mentions or impressions, without an understanding of context or tone, are largely baseless in understanding audience perception of brand, television shows, market assets, or employee and public policy.
No matter what your industry or subject matter, social sentiment analysis can enrich your brand’s audience research portfolio.
Insightful audience research is just one of the many things we think about for clients at Atlantic Media Strategies. Sign up for our weekly newsletter, the Digital Trends Index, and get in touch with us on Twitter.
