The SIFT — Using the Human Touch to Make Big Data Truly Powerful
By Jared Feldman, CEO at Canvs, and Robert Simione, PhD, Lead Data Scientist, Canvs
The rise of big data has had a fundamental impact on many sectors, from sports to dating to weather prediction to, of course, marketing.
But for all the power that big data can provide to people trying to understand and better manage systems and their companies, it faces limits too. Nowhere are those limits more evident than when trying to attach human emotions to hard numbers, to understand motivations behind actions, and to build sustainable decisions that incorporate both Big Numbers and the Big Feelings that may drive them.
Figuring out that balance has been a core challenge that our social-emotion analytics company, Canvs, is focused on. Every month, our team analyzes and classifies hundreds of millions of social-media posts about video content, extracting the underlying emotions behind audience comments about those shows and ads.
With that kind of scale, we use up a lot of computer cycles crunching numbers, tapping Natural Language Processing and other proprietary techniques to slice data and frame results with insightful summarizations.
But the process is more complicated than just turning loose a PC on a pile of raw numbers. It needs human input and just as importantly, a human understanding of how to interpret results that all our human clients can use to make deeply informed decisions.
To create a statistically valid and sustainable big-data process that actually works with the vast complications of human conversation, the Canvs team has built a process that we’ve named SIFT. That acronym stands for:
- Semi-Supervised — Algorithms are powerful things, but they can be a bit stupid. By that, I mean they’re only as good as the assumptions upon which they’re built (the site Nautilus has a great piece about bad assumptions skewing big data here). That’s why we build in a human touch to help curate results at scale. That’s vital for long-lasting quality control, and allows you to improve the product in an iterative and agile fashion.
- Interpretable — What data is actually being fed into your system? Interpretable inputs mean you can explain what your product does at a high level to a non-specialist. It also enables you to identify and debug the kinds of problems that seem inevitable when routinely grappling with huge data sets.
- Filtered — Use your domain expertise to filter your results into ready products that your clients can use to make smart, actionable decisions. Don’t bog down clients with needless details of your algorithm and its implementation. Distill the data into diamonds of knowledge.
- Translatable — Can you translate client-facing results into human language and understanding? One way to do this: create units of measurement that your clients can readily understand and use for actionable strategy and tactics. Graphically based infographics, data visualization tools and dashboards can help, too. Otherwise, your clients are receiving a pile of numbers whose import they can’t decrypt. And that’s a waste of your time and their money.
At Canvs, we’ve built a sophisticated system that parses and categorizes vast amounts of data. But what makes our system so powerful is that we marry that big data to a human touch, ensuring the system correctly interprets endlessly malleable language.
The SIFT approach has helped us stay focused on the core challenges of a product based in big data, keeping the inputs consistent and the outputs understandable.
SIFT is a vital framework for Canvs as we continue to evolve our offerings, adapting to how people use social media, how they talk about the video programming and advertising that they watch, even where and when and on what device they do it.
As it will be for many companies, this is the sort of process that will only become more important as big data continues to drive how we understand major trends and build and run our businesses in the future.
About the Author
Jared Feldman is the CEO and Founder of Canvs, the technology platform created to measure and interpret emotions. Canvs provides insight and context around how people feel towards content. Jared has grown Canvs from a small social-media startup to a multimillion-dollar company supporting clients such as HBO, NBC, Fox, and others. He’s a Social TV thought leader passionate about innovation and continuously pushing the boundaries of social media insight.