Man Meets Machine: Integrating the Human Touch into Automated Curation

Published in

Media-Nxt: The Future of Media

10 min readDec 10, 2019

SITUATION

It is a great time to be a media consumer. Content is more accessible and abundant than ever. Millions of hours of content are uploaded online every day and the volume of content being uploaded is increasing at a rapid rate. On top of produced content, the emergence of social networks and technological revolutions in the last decade have enabled consumers to create and share content of their own.

This profusion of content has granted more power to the consumer at the will of media companies. In a haste attempt for consumers’ eyeballs, both legacy and new media companies have been pushing out original content faster than ever and seeking out unique ways to further engage their audiences. All of these efforts have just added to the already overwhelming amount of content begging for consumers' attention.

To maximize return on investment on production and content creation, media companies are beginning to leverage their massive portfolios by curating content to make relevant, already existing, content easy for consumers to find. In other words, the importance of curation has presented itself to the media landscape. Players in this space have finally realized that consumer engagement does not necessarily lie in content quantity, but in quality and relevance.

Netflix and YouTube are two leading platforms in personalized user experiences. Through utilizing machine learning and algorithms, Netflix has been able to provide completely personalized homepages to their 100 million-plus userbase. This has also allowed Netflix to deliver unique value to its users by creating hyper-personalized genres of series and movies such as “Romantic Dramas Where the Main Character is Left-Handed”. YouTube’s newly redesigned homepage, which also employs machine learning and algorithms, has increased time spent watching videos on user’s homepages 20 times higher than it was just three years ago. Additionally, the platform reported that algorithmic recommendations drove 70% of time watching videos among their users.

Despite the success that both these platforms have experienced, automated curation still needs a massive amount of improvement. Automated curation has played a large role in the spread of misinformation and exposure of inappropriate content to young viewers. Both Alphabet and Facebook have already paid massive legal fees for these flaws in their systems. Additionally, a 2019 PWC Outlook reported that there is still a high level of dissatisfaction with AI-powered recommendations that consumers are receiving. Specifically, 36% of consumers feel that finding content on streaming platforms needs to be easier and only 21% of them think streaming services know what they want to watch better than themselves. Frustration with automated curation also lies in the fact that the personalization the AI is delivering is reactive, not predictive; with 30% of consumers complaining that they are recommended the same content over and over again. These frustrations are also present with targeted advertising; with countless consumers complaining that they receive the same promotion over and over again because of a google search.

To make up for the shortcomings of automated curation, many companies have begun to bring back humans into the curation process. When Apple launched Apple Music, they used human DJs in their music curation process because they believed that current technology could not handle the “emotional tasks” involved in curating music. Earlier this year, HBO launched Recommended by Humans, a platform full of video testimonials of HBO users helping other HBO users find what HBO original to watch next. Even Netflix has realized its algorithms can only go so far and launched Collections, a library of human-picked content available on some versions of the platform.

Naturally, using humans in content curation cannot deliver the scale required for existing media platforms. In fact, the original demand for automatic curation was born out of media platforms’ need for infinite and granular personalization. In an attempt to get the best of both worlds, Flipboard, a popular magazine app has adopted a model in which human and algorithmic curators coexist and help each other:

“Flipboard’s own users act as curators, adding articles to their own digital magazines for other users to read. Those curators, in turn, help Flipboard’s algorithms decide which stories and sources to recommend when users search for a given topic. But instead of stopping there, Flipboard also employs a team of human editors to make fine-grained adjustments to the output of each topic. For example, if someone looks up boating or cars as a topic on Flipboard, the algorithm might try to push out a lot of stories on accidents or crime because their sensational nature tends to get the most clicks. Human editors can then deprioritize those kinds of stories in favor of ones that are more rewarding reads.” (source: Fast Company)

However, this model has a limited amount of applications for media platforms. Any type of human involvement in curation would never be able to deliver the granularity of recommendations, or price efficiency, that automation could provide. This makes it unclear if media companies’ use of humans in curation is a permanent investment or a makeshift solution until machine learning is able to become more human itself.

PREDICTION

So how can machine learning and algorithms become more human? Or at least deliver the same value in curation as a human can? There are three fundamental changes that should be implemented into current automatic curation efforts for this to possibly happen. First, automated curation must be predictive, not responsive. Second, machine learning must become empathic. Last, curation must pull from larger and more valuable sources of data.

Automated curation must be predictive, not responsive

It is no secret that current algorithms used for personalization and curation are good at finding things that are just like other things. If Netflix sees that a user binged an entire series, they’ll recommend a similar series to binge. If Spotify sees that a user has been listening to a certain artist, they’ll recommend another similar artist. Today’s algorithms and machine learning capabilities are ‘prescribed from the past’ and rooted in context-awareness. Researcher and futurist, Genevieve Bell, has urged developers from across the globe to reengineer capacities for delight and surprise into their algorithms. Of course, personalization should still recommend more of the same, but Bell points out the main value missing from these algorithms is delight and surprise. This seems very logical when thinking of why consumers preferred human-made curation, as discussed earlier in this report. Humans can think outside the box and bring novel and surprising pieces of content into their suggestions that users usually would have never thought of themselves. If automated curation incorporated this kind of intuition, services like Spotify would not just recommend an artist similar to the one a user has been listening to; rather, they would recommend a whole new genre of music that the user could explore and enjoy.

Machine learning must become empathetic

Empathy, the ability to understand and relate to other emotions, is a key component in making suggestions to people. Human behaviors and social cues are not easy for machine learning to process, let alone appropriately react to. However, more sophisticated algorithms and advancements in machine learning are making major strides toward AI’s ability to empathize. MIT Media Lab spin-off, Affectiva, is developing emotion-recognition algorithms that measure and analyze facial expressions such as anger, disgust, joy, and fear. If this technology is able to improve and democratize, media companies will be able to understand their consumers at a much deeper level. This could even allow for emotion-based curation, recommending content for consumers that will match their mood. For example, if a user were to browse for movies on Netflix and were in a sad mood, the user could be recommended movies that are happy or inspiring, in an attempt to boost that user’s mood.

Curation must pull from larger, more valuable, sources of data

For curation to be more relevant and effective, algorithms and machine learning need to learn more about consumers. In other words, they need more data. However, algorithms are currently pulling data from the same spaces in the same manner. All of these spaces predominately lie within the digital realm, so the most valuable and untapped insights lie in physical spaces. A recent McKinsey report found that fewer than 10% of companies they surveyed currently deploy personalization beyond digital channels in a systematic way. Right now, several companies are trying to bring more value to their consumers by digitizing physical spaces. At the flagship Covergirl store, an AI interface recommends make-up to customers based on their skin tone and facial features. Traffic app Waze has GPS triggered promotions for physical retail locations embedded into their app. As developments like these continue to advance, valuable data about the consumer in the physical world can be used not only to curate relevant content experiences but also can be used to seamlessly integrate advertising into the user experience. For example, if a user has visited several makeup stores, YouTube could curate a collection of makeup how-to videos for them.

Additionally, just as data needs to be pulled from spaces in the physical world, it needs to be pulled from the social networking world. Facebook was recently approved for a patent that would allow them to use the budgeting data from a user’s friend network to create budget recommendations and benchmarks for that user. Other media platforms could follow this model by pulling the media habits of a users’ social circle into their backends when providing recommendations for that user. This would be especially effective for curating “guilty pleasures”, i.e. using network data to recommend content to users that they would not usually have heard about in social spaces.

PERSCRIPTION

If automated curation becomes able to deliver premier value at scale, then it will likely democratize like any other trend or technology. This will drastically change the dynamics of the media industry. The leaders in this space will no longer be those with the largest and most notable content portfolios, but those with the most data. The notion that ‘data is king’, is a good indicator that tech giants will likely dominate the media space. There have already been early moves made by tech giants; such as Amazon’s launch of Amazon Prime, Apple’s anticipated launch of a streaming service and Facebook’s Facebook News. Although the majority of these ventures are in early stages and lag behind media giants, they will soon likely develop valuable competitive advantages that them from their ability to provide their audiences with curated content, showing them what they want, where they want it and when they want it.

Improved automated curated experiences will also make it far easier to monetize content environments. Understanding the consumer from multiple touchpoints to deliver curated content will also make it easier and more lucrative to deliver advertising. This is not to say that banner ad prices will increase on media platforms, rather the very nature of how advertising is delivered will change itself. This is especially important considering the increasing amount of pressure put on advertisers to seamlessly integrate brand experiences into consumer’s media consumption. In response to this pressure, many brands have become creators of content, e.g. Airbnb’s Airbnb Magazine. Curated content experiences would allow media companies to create revenue streams through informing brands on what type of content to create based on the interests of the audiences they are trying to reach. Additionally, once those same brands create that informed content, media platforms can create yet another revenue stream by charging advertisers to include that content, in a relevant manner, in their curated collections to users.

Improved curated experiences could have implications across several spaces besides media, such as travel, fashion, and dining. Players in these spaces could leverage this technology to curate restaurants, hotels, experiences, etc. to their users. Specifically, Airbnb could use this technology to recommend more relevant vacation stays to their users. OpenTable could create new value for its users by curating collections of restaurants for their users, not only based on previous dining experiences but based on other aforementioned data points.

In addition to B2C spaces, this technology has major implications in B2B and government spaces. Businesses can leverage this technology to have curated news streams that inform them of current events that relate to their business or industry. Local governments could utilize this technology to have curated collections from council and board meetings so that they can efficiently process and understand issues being posed in the local community. Curate, an early-stage startup, is developing these types of processes to help businesses and local governments become more efficient.

While it’s exciting to imagine the potential implications automated curation may have, it is also important to consider the barriers that could prevent it from coming to fruition. The largest barrier this trend faces is data privacy regulations. Legislation such as GDPR has been implemented into governments to protect individuals’ data. Without access to a large pool of data about individual users, the algorithms and machine learning behind the curation technology will not have sufficient context and information to deliver relevant and useful recommendations. Another major barrier to this trend is the deconsolidation of media and/or tech giants. Automated curation heavily relies on scale to deliver value to the user. Furthermore, machine learning is able to improve its functioning by receiving feedback. If media and tech giants are broken up, the individual AI and algorithmic programs will likely not have enough data to develop properly.

There are several kinks to be worked out of current curation processes and the capacities of current machine learning technology are unknown. However, what can be said for certainty is that there must be some sort of mechanism developed to act as a compass to navigate consumers through the stormy waters of media.