Netflix Data Science Interview
Netflix CEO — Reed Hastings predicts 15B$ in revenue this year.
On May 9, Netflix launched its own research website. This highlights the focus Netflix has on Deep Learning and Data Science. The site is extremely well designed showing vertical classification of the different areas that Netflix research works on along with the horizontal business areas where Data Science is deployed at Netflix. It has some great articles with everything from video encoding to A/B testing where they use Data Science. I found the website to be very comprehensive making it a go to destination for things Netflix Data Science from different verticals to jobs.
To maximize the impact of their research, Netflix does not centralize research into a separate organization. Instead, they have many teams that pursue research in collaboration with business teams, engineering teams, and other researchers. From our publications we can deduce that they are focused on the applied side of the research spectrum, though they do pursue fundamental research and think that has the potential for high impact, such as improving our understanding of causality in our data and systems.
Netflix moves quite fast. There is one phone interview with the recruiter and another detailed one with the hiring manager. There are two onsite interviews with around 4 people first time (data scientists/engineers) and 3 people (higher level execs) second time. There is a mix of product, business, analytical and statistical questions. Statistical questions mostly revolve around A/B testing: hypothesis testing. There are a couple of SQL questions too. Analytical questions usually includes a hypothetical problem to analyze and metrics to evaluate product performance. Higher level executives mostly focus on background and past experience.
Important Reading
- Netflix Research Blog: All Articles
- Deep Learning for Recommender Systems: Talk Slides
- Reliable ML in the Wild Workshop (ICML 2017): Making ML Reliable at Netflix
AI/Data Science Related Questions
- How would you build and test a metric to compare two user’s ranked lists of movie/tv show preferences?How best to select a representative sample of search queries from 5 million?
- Given a month’s worth of login data from Netflix such as account_id, device_id, and metadata concerning payments, how would you detect fraud? (identity theft, payment fraud, etc.)
- How would you handle NULLs when querying a data set? Are there any other ways?
- What is the use of regularization?What are the differences between L1 and L2 regularization, why don’t people use L0.5 regularization for instance?
- SQL queries to find time difference between two events given a certain condition.
- Given a single day with a large sample size and a significant test result, would you end the experiment?
- What do you know about A/B testing in the context of streaming?
- How do you prevent overfitting and complexity of a model? How do you measure and compare models?
- How do you know if one algorithm is better than other?
- Elaborate on the recent project you developed for your company.
- Why do you use XYZ method? Elaborate on how to improve content optimization?
- What technology or item that most people feel will be obsolete in the future do you not agree with?
- Why Rectified Linear Unit is a good activation function?
- How should we approach attribution modelling to measure marketing effectiveness?
- How would you determine if the price of a Netflix subscription is truly the deciding factor for a consumer?
- If Netflix is looking to expand its presence in Asia, what are some factors that you can use to evaluate the size of the Asia market, and what can Netflix do to capture this market?
- Say the CEO stops by your desk and asks you whether or not we should go into an untapped market. How would you determine the size of the addressable market and the factors the Netflix should consider before deciding to enter the market?
Reflecting on the Questions
The data around Netflix questions is sparse. The high level questions resolve around A/B testing, recommender systems and foundational knowledge questions around regularization and activation functions. This is different from the other companies we have looked at previously where focus was more foundational. All job openings are usually senior level. Good experience combined with good preparation can surely land you a job at the largest international evergreen content cinema in the world.
Consumable List: Netflix Data Science Interview Questions
This article was also featured on KDnuggets: https://www.kdnuggets.com/2018/06/netflix-data-science-interview-questions-acing-the-ai-interview.html
Thanks for reading! 😊 If you enjoyed it, test how many times can you hit 👏 in 5 seconds. It’s great cardio for your fingers AND will help other people see the story.
Subscribe to our newsletter here. We are building a new course to help people ace data science interviews. Sign up below to join the wait-list!
