How to detect the most promising YouTubers using data science

From a Product manager’s perspective…

This article would have not been written without the work of our talented data scientist Jean-Jacques Simeoni. Special shoutout to him and to all teams that have helped build this feature.

Since its creation in 2016, Jellysmack has experienced a meteoric rise and quickly established itself as the global creator company that detects and develops the world’s most talented video creators. Our proprietary video data and optimization technology drive massive social audience growth for creators, unlocking new revenue streams and amplifying monetization.

Of course, the first step of this success is to detect the most promising creators. Let’s talk about YouTube for instance. There are tons of creators on this social platform, both casual and professional. When I say “most promising creators,” it doesn’t mean promising for YouTube, it means promising for additional social platforms. Jellysmack is all about taking video content from one social platform to multiple.

At Jellysmack, we have a whole team and a proprietary tool dedicated to detecting promising creators. As a lead product manager I’m in charge of this scope. In this article, I’d like to use a concrete example to highlight how a strong collaboration between the product manager and the data scientist can lead to game-changing features.

The rising stars detection idea is born

One of Jellysmack’s co-founders has a special power. He’s able to detect promising talents on YouTube that will perform well on other social media platforms. And you know what? He has proven that with many of our onboarded creator partners who are performing well today.

During a call with him, I was trying to understand how his magic works. He told me:

“Ryzwan, the most interesting creators for us are not the big ones on YouTube with millions of subscribers. We should be focused on the small ones with high potential. That’s why I try to find the rising stars.”

And he was right. Smaller YouTubers need more assistance in the beginning, and there’s more opportunity to make a large impact on their career trajectory... The need for a programmatic way to detect rising stars was born.

Here’s where the classic Product Manager ideation journey starts:

http://www.mayadaysmith.com/

Over time, we created several versions of this feature trying to detect the rising stars on YouTube:

• V0: with a bunch of complex rules based on several metrics — Dropped after an audit
• V1: with a simple rule based on the subscriber growth — Dropped after an audit

Yes, we made mistakes but we learned a lot along the way. The most important question here is WHY? Why do some creators have high potential and others don’t? Detecting rising stars is not easy. We’re not talking about telling if it’s blue or red. We’re talking about analyzing the creators’ presence on YouTube and predicting their future performance. I said “Predict?” Yes, that’s exactly the word here!

While I was reviewing why those two versions had failed with a data scientist from my team, Jean-Jacques Simeoni said:

“We should use data science to predict how the creators will perform over time to detect the rising stars.”

Good idea, yes! But predict what? What do we want to see from those rising stars?

For this V2, creating a useful feature was crucial. So let’s start again from the beginning and discuss with the stakeholders what a rising star is for them. What do they expect from a rising star?

This time all stakeholders were aligned on the definition of a rising star. Some creators have a really high potential to grow but they need more time and more money to do so. Jellysmack can help them grow faster and unlock their full potential. We decided that rising stars are small creators who can sizably grow their revenue on YouTube within the next 6 months.

Let the data scientist do his magic

The business teams are not always aware of the power or capabilities of data science. So once the need was correctly clarified, we decided to try several POCs (proof of concept) on our side and evaluate them. Finally, if the results are encouraging, we present them to the stakeholders.

Data scientists don’t work as developers do. As product managers, we must be aware of that or it won’t fit well with the workflow. Here are the 4 steps we followed to build this feature using data science.

First step: the target definition

KPI, KPI, and KPI! I’m sure my team is tired of hearing this word but yes again: KPI. What do we expect from this new feature? What’s the target to reach, and how will we be able to follow it?

The KPI and the target we defined with Jean-Jacques Simeoni before building the feature was:

Second step: the investigation

All right, this is the most important step.

The investigation time has been used perfectly to clarify the needs with the stakeholders and search for existing data science models on the web. Finally here were the outputs:

• What’s a small creator? As said before, the rising stars might have the potential to sizably grow their revenue but they need help to unlock it. Jellysmack is here to handle that, also on the financial part. At this step, we have defined the creator’s maximum monthly revenue until which Jellysmack is able to significantly grow it.
• How can we estimate the YouTuber’s revenue if we don’t have the info? Using the data we have acquired by our creator partners and dozens of our own popular social channels, we can estimate the YouTube revenue.
• How can we predict the revenue in the future? Using an existing data science forecasting model, we can predict future revenue based on the past performance of the creator.

Third step: the POC(s) creation

Now it’s time to dive in, but again: data scientists don’t work as developers do. As product managers, we should keep that in mind. This step might not occur as planned. Sometimes, the POC creation will take more time. Sometimes we will need to create and compare several POCs. Sometimes it will be a NO GO. But let the data scientist do his magic, it will be beneficial, trust me.

Fourth step: the POC evaluation

Once finalized, Jean-Jacques Simeoni presented how the POC works. After a few iterations here is the summarized schema of the last POC:

So, we have the POC, but what about the KPI? As a reminder the KPI was defined as: 80% of the creators considered to be rising stars by the algorithm should be actual rising stars.

To do so, for 200K+ creators Jean-Jacques Simeoni:

• Turned back the clock 6 months ago
• Ran the newly created model to predict those creator’s revenue for the next 6 months.
• Compared the revenue prediction vs. the reality (estimated revenue based on monthly views)

Finally, we asked the stakeholders to evaluate a sample of creators. In the sample, they were either rising stars defined by the model but also not rising stars. Then, we compared both results.

KPI reached, we can move on. Time to pitch our project to the stakeholders! The goal is to convince them. Non-tech people don’t always trust data science when it’s complex. To persuade them of the power of the algorithms, we have to show trustworthy key figures. We must also focus on making the data transparent and easy to understand.

To do so, I introduced the context, the needs, the stakes, and the data scientist presented how the algorithm works. I then finished with the evaluation key figures and the next steps to get the feature released. From my experience, allowing the data scientists to showcase their hard work lets the stakeholders understand the tremendous amount of effort that went into the model.

YES! We can breathe. After the presentation, the stakeholders were convinced. It was a GO! A few steps left:

• Create documentation of course. Both functional and technical
• Industrialize the algorithm by working with the Infrastructure team
• Work on the backend and the frontend parts of the feature

Here we are! The feature is now available on our tool.

The feature has been released but it’s not finished. We’ve set up processes to continuously evaluate the functionality using user input and periodically testing the prediction model as more and more data is collected. Based on what we’ve gathered, we’ve already planned improvements for V3.

Data empowers your features

Product managers have so much data to play with. But working with a data scientist allows you to unlock the data’s full potential.

Raw data is useless. We need to deep dive into it to take full advantage of its power. Oftentimes, data science allows you to suggest opportunities the stakeholders can’t even imagine. That’s the strength. But we can’t forget key elements like defining a KPI and continuous reassessment. Following a proper process allows you to achieve the highest results possible.