How to hack and achieve your ROIs from mining consumer insights
What is consumer insights from big data?
The data source for collect big data can be from any public website that can be accessed from a web browser. Key sites where useful insights are usually found are social media, forums, news website and blogs. These are sites where the end user posts opinions on a product/service, ask questions and discusss with other on a topic.
We use these insights to seek out answers on what consumers and the general public is thinking. To make better sense of this raw data, we use machine learning algorithms to clean up and process the data into more useful insights.
The tedious and expensive process of sending questionnaires and doing focus groups are gone
In the pre-20th century, the only way to get feedback of the masses is through laborious questionnaires and focus groups. With big data today and greater venues for end users to post their opinions, there is less need today to do so. Feedback can be obtained instantaneously from the users. If users are not satisfied with a product or a service, they can go to public review sites and leave negative feedbacks there to voice their displeasure. With the prevalence of the internet, this user feedbacks have the probability to go viral and be made influential.
The right way of getting consumer insights going into the future is to use a combination of big data and machines. This should be done in an automated manner. Processes should be set up to collect the widest sources of consumer insights possible and to repack this valuable insight into a convenient form that the masses can utilize. The user of the platform can now easily gain full insights on what the aggregated consumer thinks.
Breaking down the different insight features from different sources of data
Social media
Social media provides a large rapid firehose of insights on almost any topic under the sun. However, with large volume comes greater noise. Through social media, the demography of the users is broad. They can comprise of insights from your own organic audience base and other audience bases.
Features
- Real-time feedback on any topic under the sun
- Collect raw unsolicited feedback
- The most ‘latent’ form of insights, had the tendency to show early shifts in consumer trends and preferences
Public websites
Public websites comprise of review sites, blogs and forums. The intent of authors on public websites are more intentional than unsolicited social media opinions. It is often used to communicate thoughts to a specific audience base, with the intention to persuade. It is the medium which thought leaders and subject matter experts use to share opinions and knowledge.
Features
- Get targeted, more detailed insights
- Get insights from a group of experts, rather than a broad general audience base
- Understand the news coverage and mentions on your brands
We need to clean the data before we can put them for real uses
Raw data by itself contains a lot of noises. To make the data usable to the end users, we need to make good sense of the data. Rather than cleaning the data laboriously by hand, we use some state-of-the-art algorithms to do so, which is through a mix of natural language processing and classification methods.
Natural language processing (NLP)
Raw data posted online are often noisy, with slangs, spelling, grammatical errors, colloquialism and abbreviations. Through our pre-trained text libraries, we attempt to correct and clean up these data automatically before we pass them into our analytical algorithms. Key NLP algorithms we use are sentiment analysis and emotion analysis, which tracks the feelings of users over time.
Classification
Classification is employed to clean up data, intellectual to inputs given by the user. No matter how ‘trained’ a library can be, it will often not be able to sort data with a 100% accuracy because of the many nuances. To address this, we need to use classification algorithms. The expert end users will manually tag the data and identify which are positives and negatives. The machine will observe this and clean the rest of the data. This classification method can significantly improve the accuracy of the insights.
Enabling the user to react fast
Being able to be the first to react to insights is a competitive edge to the decision maker. Stale data are often non-effective data. We all want to be the first to react to material information. There are features in the dataset which the end user are often interested in — volume spike, sentiment change, trending hashtags. Alerts can be set on these features to notify alert the user if any of these features go off, which can be in the form of either mobile push alerts or email alerts. For a key decision maker, he/ she will want to be aware of any important sentiment shifts in the company, competitors and industries. Conventionally, companies would hire a team of social media analysts to track all online conversations on their company manually. This process should be replaced by automated means in listening 24/7 smartly.
The ROI of leverage consumer insights comes in 3 main forms
1. Brand and campaign management
The most direct use case is brand management, where organizations and influencers can hear what the web-at-large is talking about them. What do people like or dislike about them? What the conversation topics that are surrounding them?
On the marketing campaign front, companies need a data-driven way to quantify the unstructured data surrounding their marketing campaigns. It will be useful to know what campaigns best resonated with their target audience? Which went best, which went bad? Companies can then best learn from their past campaigns.
ROI:
- Find out the context where the brands are mentioned, to craft more effective future campaigns
- Adjust campaign tactics on the fly by tracking real time user responses
- For traders, unlock a well of unstructured data on the companies they are tracking to gain an edge over other peers and be alerted in real time, essential for short-term traders
2. Audience analysis
Knowing who is talking about the brand is equally as important as what is being said of the brands. This translates into companies knowing about their audience demography. In turn, this information can be used to better drive tailored future marketing campaigns to address each demography segment. Different marketing messages will appeal to different groups of people.
ROI:
- Craft better-targeted messages for each demography, leading to increased market share and greater revenue
- Able to unlock new audience that the company was not previously aware of
3. Industry and trend analysis
It is imperative to aggregate the big picture trends on the industry that drive consumer behaviours, for any strategic decision maker. Organizations can then form the complete picture view of what drives their users. Companies can understand what are the trends and driver shifts over time, and the motivations and behaviours behind their users. Through a better understanding of the competitive landscape, we can understand the strengths and weaknesses of competitive offerings, track the share of voice and track any new entrants.
ROI:
- CEOs/CMOs can stay ahead of the curve in understanding what drives behaviours of its audience
Summary of effective use cases
UserUse CasesROICEO/Consultant
- Understand industry shifts over time
- Understand consumer motivations and behaviours
- Understand the strengths and weaknesses of competitors
- Detect and capitalize on industry trends
- Evaluate and adjust to competitive landscape
CMO
- Understand consumer’s path to purchase
- Visualize existing audience
- Find a new audience
- Fine tune audience segmentation and message
- Measure and boost campaign performance
Communication/Influencer
- Track and react to brand reputation in real time
- Build a stronger brand narrative
- Provide customer care more quickly
- Create positive engagement
- Able to command better brand premium and fees
Trader
- Analyze sentiment shift of companies in real time from obscure yet impactful sources
- Overlay a new source of alpha source which is almost impossible to obtain trivially
- Run more effective short-term trading strategies (i.e. market neutral, stats arb style)
For more posts like this, visit Latent App’s blog.
