Understanding the “YouTube rabbit hole”
How watching one video turns in to a three hour video-binge
We’ve all been there, it starts innocently enough by watching a how to video, perhaps. Before you know it, it’s 2 a.m. and you’ve been watching reviews for a new motorised toothbrush for the past three hours.
Colloquially known as the YouTube ‘Rabbit hole’ which describes the process of watching endless YouTube videos but a closer look in the the YouTube recommendation algorithm will reveal how this happens.
One of the issues about understanding this recommendation algorithm is that Google are very cagey and vague over this area. It comes down to data science and artificial intelligence. The best piece of journalism on the rabbit hole is from New York Times journalist Kevin Roose so I will credit him to some of the research throughout this blog.
Artificial Intelligence
In 2015, Google’s deep learning artificial intelligence research team Google Brain, began rebuilding YouTube’s recommendation system around neural networks, a type of A.I. that mimics the human brain. In a 2017 interview with the Verge, a YouTube executive said the new algorithm was capable of drawing users deeper into the platform by figuring out “adjacent relationships” between videos that a human would never identify.
Essentially, one neural network filters videos to see if they are good candidates for the viewer’s “next up” selection (based on the user’s history, and what similar users have watched.)
Meanwhile, a second neural network ranks videos by assigning them a score. This is based on factors which are not revealed, entirely: although a video’s newness and a channel’s frequency of uploads are both mentioned.
The idea is not to identify “good” videos, but to match viewers with videos that they want to watch. The end goal being that they spend as much time possible on the platform.
The new algorithm worked well, but it wasn’t perfect. One problem, according to several of the current and former YouTube employees, is that the A.I. tended to pigeonhole users into specific niches, recommending videos that were similar to ones they had already watched. Eventually, users got bored.
The Reinforcement Algorithm
Therefore Google Brain’s researchers wondered if they could keep YouTube users engaged for longer by steering them into different areas of YouTube, rather than feeding existing interests. They began testing a new algorithm that incorporated a different type of A.I., named reinforcement learning.
The new A.I., known as Reinforce, was a kind of long-term addiction machine. It was designed to maximise users’ engagement over time by predicting which recommendations would expand their tastes and get them to watch not just one more video but many more.
Algorithm is behaviour driven
According to YouTube, the following user behaviors are part of what guides the algorithm’s choices:
- what people watch or don’t watch (a.k.a. impressions vs plays)
- how much time people spend watching your video (watch time, or retention)
- how quickly a video’s popularity snowballs, or doesn’t (view velocity, rate of growth)
- how new a video is (new videos may get extra attention in order to give them a chance to snowball)
- how often a channel uploads new video
- how much time people spend on the platform (session time)
- likes, dislikes, shares (engagement)
- ‘not interested’ feedback (ouch)
Constantly Tweaking
YouTube’s recommendations system is never set in stone and is constantly changing due to advertiser demands, regulation and increasing user engagement. The company makes many small changes every year, and has already introduced a version of its algorithm that is switched on after major news events to promote videos from “authoritative sources” over conspiracy theories.
For years, the algorithm had been programmed to maximize views, by showing users videos they were most likely to click on. But creators had quickly learned the system, inflating their views by posting videos with click-baited titles or choosing eye-catching thumbnail images. YouTube’s executives announced that the recommendation algorithm would give more priority to watch time, rather than views. This allowed YouTube to have a better understanding of user’s consumption and also improving their recommendation algorithm.
The Criticism
YouTube has defended its video recommendation algorithms, amid suggestions that the technology serves up increasingly extreme videos.
But the company’s new managing director for the UK, Ben McOwen Wilson, said YouTube “does the opposite of taking you down the rabbit hole”.
But YouTube has never explained exactly how its algorithms work. Critics say the platform offers up increasingly sensationalist and conspiratorial videos.
Mr McOwen Wilson disagrees.
“It’s what’s great about YouTube. It is what brings you from one small area and actually expands your horizon and does the opposite of taking you down the rabbit hole,” he says.
Follow Me on Twitter
Sources Used:
https://www.scientificamerican.com/article/youtubes-recommendation-algorithm-has-a-dark-side/
https://www.bbc.co.uk/news/technology-49038155
https://www.theatlantic.com/technology/archive/2018/11/how-youtubes-algorithm-really-works/575212/
https://blog.hootsuite.com/how-the-youtube-algorithm-works/
