How Netflix deal with big data

Satya Kunapuli, head of data engineering and analytics at Netflix, recently visited Australia. He shared how his team works with large-scale data processing to power machine learning algorithms. Their extensive use of data to test, learn and iterate supports paid media marketing outcomes.

Netflix’ stunning subscriber growth

Netflix has over 100 million subscribers in 190 countries, and gains 1.5 million new subscribers every year. “We went global quickly to ensure that our services scaled, and to learn about users viewing behaviours as quickly as possible.”

Their first original show, House of Cards, was produced four years ago. Since then Netflix has become the premium destination for exclusive content, with over 400 original shows to date. “The biggest reason for growth is original content.” This growth represents a huge monetary investment and an equally huge amount of data to explore.

Machine learning and machine buying

Contemporary testable processes and experiments are now overshadowing traditional marketing metrics. As the traditional marketing correlative approach doesn’t suit the scale that Netflix now operates at. With extensive global campaigns to manage and perfect, they instead use large-scale data processing and machine learning algorithms to not only learn about their market, but to decide:

• What advertising creative to show customers
• Which customers to show the creative to, and
• What programmatic channels to invest in (where the advertising buying process is automated).

During the learning phase, they essentially want to zero in on the the factors that affect conversions and signups, and adjust them to be more successful. They use a range of techniques to do this.

Trusted data analysis techniques

Thoughtful descriptive analytics

Metric innovation research is undertaken to understand what conversion windows work best for a title. Data engineers look at data, set conversion windows, devise their own metrics to interpret demand, then calculate if the metric matches what’s being watched. If there’s low demand they shift money off the title. If demand is high but conversions are low, they’ll consider investing more money marketing the title.

Regional quasi-experiments

This observational study is used for regional or programmatic channelling experiments, which reveal patterns in fundamental conversion and response rates. It’s possible to see the incremental effect of one combination of factors (a mix) versus another. This provides a sense of what incremental sign on is in the region tested, over a period of time.

A/B testing

A/B testing has been a core technique for Netflix. They use a control group and a test group, which provides a clear view on the causal effect of marketing, as long as the groups are randomised (to prevent bias). A/B testing, while effective, is viewed as time consuming, and hasn’t provided decisions on the scale Netflix want. So they’ve turned to incremental based algorithms for more scalable results.

Incremental based algorithms

Qualitative decisions like, which titles and which markets to invest in are better made by humans. For more complex decisions like, which channels and audiences to target and with what creative, are where Netflix invests in automation. They need to calculate how best to market the right title, to the right person, at the right time, across regions for a huge amount of titles.

How advertising is served at scale

Programmatic advertising is a process where Netflix and other advertising buyers bid on an impression. If the bid is won, their creative is instantly displayed on the publisher’s website. Whichever execution is served is based on the extensive data exploration, and testing that’s taken place for a particular region. This process is measured and iterated upon to achieve the desired conversion or sign up goal. “This whole thing has to happen at a ridiculously high scale.”

More shows to watch

Satya sees the future of analytics at Netflix being a balance between man and machine working together. Where all the decisions at scale can be automated so campaign managers can save time. The goal is that everyone as a subscriber has something interesting to watch at different times.

Get in touch with us to talk through your big data and experience design challenges!