Looking at Current News Content Recommendation Systems
Content recommendation plays an important part in the online publishing ecosystem. It provides a reader with suggestions about what to read next and has a revenue stream attached to it, whether that means additional revenue from internal circulated content or ad revenue through external sponsored content.
In fact, according to Parse.ly, on average, recommendation systems provide a 3.2% uplift on page views. With that said, content recommendation systems, today, come in many forms — some bad, some good, some technologically advanced, and some manual. I have grouped content recommendation systems into three main categories: in-house, third party, and native advertising.
Many publications opt to use in-house systems to serve related content. However, the way the in-house system works varies greatly from outlet to outlet— from a completely manual related content tagging system used by most publications, local and mass, to an automated recommendation system that takes into account context and user behavior.
It is interesting to me that many publications today still use a manual tagging system, which speaks to the industry’s slow adoption of new technologies and distrust in automation. In my opinion, a manual related content system leaves a lot of room for error and is a meaningful drain on editorial resources. For starters, editors have to have a perfect memory to know which past content relates to the current article. In speaking with editors, they admit to often forgetting to link to content that users would find helpful as a next read. That “leaves clicks on the table.”
But I do understand publications’ concerns around a fully automated solution. From an editorial perspective, the last thing you want is an article about dieting to be linked to an article about cryptocurrency. The obvious solution here is to implement a semi-automated solution where the editor has the flexibility to change the automated recommendations.
But most publications simply just do not have the technical resources to develop their own systems and maintain the algorithms so problems like the above don’t occur. Companies such as the New York Times and Washington Post have the resources to develop and implement automated recommendation systems, but your local county publication does not. Enter third party recommendation systems.
Over time, technology companies realized that most publications do not have the internal resources to build strong recommendation systems and created related content systems to provide as a software as a service (“SaaS”). This SaaS offering is usually priced as a revenue share on revenue per click or a monthly fee determined by the size of the publication. A few providers in this space include Contextly, Bibblio, and AddThis.
The competitors in this space all go to market with a similar product. Sure, they compete on price and sure they say they have the best algorithm; however, the core end product usually looks the same. A set of automatically generated recommendations that appear in the ad space of the article. And even though each of the competitors’ algorithms are different — some weight context over behavior, some use different recommendation algorithms such as LDA or TFIDF — each algorithm is generally accurate and provides meaningful related content for the reader to explore.
In addition, third party providers have innovated to differentiate from competitive solutions. For example, Contextly created a “Follow This” feature to send readers new updates on a story they are interested in, and AddThis created a recommendation system that renders related content as readers scroll to a set point on an article web page. This type of creativity not only provides new, unique experiences for readers, but also allows providers to own more real estate on a publications site which increases the probability of reader engagement. In summary, third party content recommendation tools are a solid choice for publications who do not have the tech resources to build its own system but still want to take advantage of an automated solution.
The final bucket of content recommendation systems include Taboola, Outbrain, and RevContent. These providers operate in a similar structure as other third party recommendation systems but render low quality recommendations. In fact, a lot of these recommendations can be considered click bait. Effectively, the related content act like ads. That said, because the recommended content is eye catching, it does generate clicks and therefore generates revenue for the publication.
But the extra revenue generated from Native Advertising comes at a cost. First, the content recommended is random, it is rarely related to the loaded article. Using the above image, an article recommendation about underwear is placed next to the paragraph about Elon Musk’s compensation plan. This is distracting to the reader and does not enhance the reader’s ability to better understand the active topic. In addition, the format and style of recommendation takes away from the publication brands and look/feel of the website. It is obvious that the widgets are from a third party and do not feel native at all. So while native advertising tools do add some monetary value, publications that use these tools are sacrificing brand value and reader experience.
While current forms of content recommendations work well, there is always room for improvement. At Hindsight, we believe the structure of content recommendation itself is flawed. All the above tools recommend content in the ad space of an article. The location of these recommendations take the reader away from the article and is not aligned with the typical reader experience. We are working on an experience that is rendered by the reader, within the article, at specific interest points. We provide term specific recommendation, so when you come across Elon Musk in an article, we recommend you more articles about Elon Musk and when you come across Tesla Model 3 in that same article, we showcase more content around the vehicle.
This is point-of-interest content recommendation built for the reader that responds to the aspects of an article a user is most engaged with. For the publisher, it’s an additive format that doesn’t replace existing recommendation solutions but complements them. To learn more, please reach out to me at hersh@hindsightsolutions.net or visit our website.
Originally published at medium.com on May 21, 2018.
