How to prepare for Machine Learning Interviews from a Non Computer Science Background ?

The goal of this article is to provide the reader with a comprehensive study plan for the second interview of any position like Data Scientist, Data Analyst or a Machine Learning Engineer in a typical Tech company. The article starts by dividing the the Machine learning (ML) interviews in the three major areas a candidate is usually tested on and then it lays out all the resources that can be very useful in helping a candidate prepare for the interview. The article ends with my own observations and experience in this journey of learning Machine Learning and Computer Science fundamentals.

Sierra View Point (Milpitas, CA), Source- WWW

1. Introduction

The purpose of this article is to help the reader prepare for Technical interviews focused on Statistics and Machine learning (ML) Fundamentals in a standard Tech company . Preparing for Tech company interviews is a very time consuming process as the bar for hiring a candidate in a Tech company is very high. Hence, the candidate is required to provide the interview panel with a lot of data points in order to meet that high bar. As a result it helps if a candidate is very well prepared when he or she is taking the interview. A large part of this interview preparation process is to have a clear understanding on what is expected from a candidate and access to the right resources that can help the candidate reach those expectations as quickly as possible.

This article is written for those candidates who do not have any prior experience in interviewing for a Tech Industry job and who are not from a computer science, programming or a STEM background. Four years ago I was one such candidate, an Econ major doing her first job in the banking industry in the Mid-West USA. I had about one year of work experience when I started looking for a job in the Tech industry of the SF Bay Area. After failing a few interviews I realized that I needed to have a different category of information in order to clear these tests. Over the course of years I have tried various resources like medium blogs, online courses, reading books etc. to acquire the skills and knowledge required for the ML fundamental interviews and wasted a lot of time chasing the wrong resources. I did not have a friend or a colleague who had undergone a similar journey and who was in a position to provide me with the so necessary tips and best practice based guidelines. This article is to help a candidate like me from needlessly meandering into the massive rabbit hole of random online courses and YouTube channels that might not be the best utilization of one’s time.

Statistics and ML fundamentals are tested in the second stage of the interview process after a candidate has cleared his or her Technical phone screen. Please see my other article to get more information on a study guide for the Technical Phone Screen component of a Tech job interview. In some companies, the Technical Phone screen also involves ML and Stats Fundamentals. Assuming that the reader has a decent exposure to mathematics and Calculus during the school days but is a little bit rusty on the things he or she learnt back then, the following resources can be a very good refresher if he or she is preparing for a Data Scientist or a Machine Learning Engineering (MLE) opportunity. Per my experience, Data analyst roles do not have a strong focus on this subject. Please see my other article to get a sense of various technical job positions people usually pursue in the Tech industry.

2. Common Subjects Tested in ML Interviews

The reader needs to brush up on three fundamental areas for successfully clearing the second stage of the Technical Interviews.

1. Linear Algebra
2. Undergraduate Statistics

a. Probability and Distributions

b. Statistical Inference and Hypothesis Testing

3. Machine Learning Fundamentals

a. Linear and Logistic Regression Models

b. CART Models

c. Neural Network and Other Deep Learning Models

d. Unsupervised Learning Algorithms

In the remaining portions of the article, I will attempt to direct the reader to resources that I found very useful over the course of my career in the Tech industry.

1. Linear Algebra

Linear Algebra is the fundamental building block of any ML algorithm and it is amazing how much of it one can forget after leaving school for a few years. I remember feeling very stupid when I was asked a simple question- “What does a rank of a matrix mean?”, in one of my very early Tech interviews. I was very close to bumping my head on the screen as that’s something I knew that I had learnt at some point of time in my past and had no recollection of it at the time of my interview. I was not the first person in a similar situation, most of us can be caught off-guard by some very fundamental concepts of Linear Algebra. I used the following resources to prepare for Linear Algebra based questions that are often a part of phone screens in some Tech interviews. This also helps a candidate in demonstrating a thorough understanding of the fundamentals of ML.

a. Gilbert Strang’s Lecture series in MIT open course

After watching this series I wished I had known about its existence during my graduate school days when I struggled with the metrices and the determinants. I am yet to come up with a more high quality material on the subject. However, each lecture in this series is an hour long video and one needs to have a lot of time in hand in order to truly learn from this master. I have skipped a few lectures and focused on the ones that are most relevant.

b. Khan Academy

Khan Academy’s videos also cover the various concepts of Linear Algebra with an amazing clarity. The videos are of shorter duration that affords learning-on-the-go for folks with a busy life.

The candidates need to have a good understanding on the following areas of Linear Algebra at the very least. The deeper the concepts one is able to develop, the better it is for them.

1. Concept of vectors
2. How to add and subtract vectors?
3. How to solve a system of Linear equations and under what conditions a solution exists ?
4. Matrix Multiplication
5. Determinants and Eigen values/ Eigen vectors
6. Cofactors, Cramer’s Rule, Orthogonal vectors, Cramer’s Rule
7. Various Types of vector Spaces and basis of a vector
8. Vector Projection
9. Why is a Linear Regression an orthogonal projection ?

Some ML depth interviews ask a candidate to derive the mathematical formulations of an ML algorithm and an understanding of the above concepts are crucial in demonstrating one’s grasp on any area of ML.

2. Undergraduate Statistics

A very advanced knowledge of statistics is a must-have for clearing Data Science interviews where there is almost always a section dedicated towards Statistics and Econometrics. If someone is rusty on what was taught in school a few years ago, then one can consult the following sources to brush up one’s understanding of fundamental statistics and Econometrics.

a. Prof John N. Tsitsiklis’s Lecture Series in MIT Open Courseware

This is another master-piece available for free for anybody who wants to learn Statistics. After watching Prof Tsitsiklis’s lectures and reading the first two chapters of his book, I realized that there were fundamental gaps in my understanding of how statistical hypothesis testing works and I acutely felt it during my graduate school days. It drove me mad and made me avoid statistics, unconsciously. I am really lucky to stumble upon this gem, available for free. If the reader is short on time, then he or she can focus on the first two chapters of his book. This will empower the reader with all the basic information needed to answer complex probability problems that are often asked in Technical Phone Screen interviews of both Data Science and MLE positions. All those “tossing a coin” and “rolling a die” problems will be a piece of cake after going through this.

Recent Data Science interviews are combining coding and statistical interviews by asking a follow-up question where the candidate is often asked to verify their answer to a simple die-throwing or coin tossing problem by implementing a simulation algorithm. In such interviews, the candidate is expected to use a function like np.random.choice and generate numbers in the range (1,6) or (0,1) some fixed number of times and then implement the situation mentioned in the probability problem algorithmically (using python code). So it’s good to practice both the theory and the coding part of the probability problems.

3. Machine Learning Fundamentals

I have used a wide variety of resources over time to improve my understanding in this area. I will list the resources, I used, in a particular sequence, which, I believe, will help a reader, with very limited/rusty prior knowledge on this topic, the best.

a. Introduction to Statistical Learning

This is a book written by two professors with ample industry experience in ML. It’s a wonderful amalgamation of the basic concepts of the classical ML models that are still very important for the Data Science and MLE roles. The book has plenty of problem exercises with solutions written in both Python and R codes. The concepts are explained in a concise manner that provided me with just the level of clarity and details I needed when I was trying to understand Machine Learning for the first time, many years ago. After reading this book, the reader will have a very good grasp on the fundamental ML Concepts like:

1. Bias Variance Trade-off
2. Supervised and Unsupervised Learning
3. Linear and Logistic regression Models
4. CART Models
5. Some Bayesian Methods

Per my experience, reading this book should be enough to clear the ML fundamental questions asked in a standard Data Science interview.

b. Introductory Econometrics by Jeffrey Woolridge

This is a standard econometrics Text book familiar to anyone with an undergraduate degree in Economics. This book is probably the best resource that one can use if one wants to develop a detailed understanding of the inner workings of a Linear/logistic Regression and Time Series models. These are also the fundamental building blocks of most Causal Inference models- which are often a must requirement for Data Science interviews.

This is a very fat book with many chapters, usually completed by a student over the course of one or two semesters. Hence, it requires considerable time investment from the reader. After reading this book one would develop a concrete idea on concepts like:

1. What are the assumptions of a Linear Regression Model?
2. How to test these assumptions and what are the work-arounds if these assumptions are violated?
3. What is standard error and why is it so important ?
4. How do dummy variables in an equation work ?
5. What are interaction terms and how can one mathematically interpret them?
6. How does an Instrumental variable work ? What is omitted variable Bias ?
7. What is multinomial Logistic Regression?
8. How do time series processes work and what assumptions need to be tested before applying them to a business problem?

A lot of business problems can be approximated by Linear formulations. Classical ML practitioners often start with a Linear approximation to develop the first sense of bias and variance existing in the data. Often times, those formulations skip basic checks of the assumptions of a Linear model. Hence, a good understanding of this area is very helpful, no matter which job one is pursuing.

The following resources are mostly useful for candidates pursuing MLE roles or a very technical Data Science position. The first two resources, should be enough to help a reader clear Data Analyst and most Data Science interviews

c. Andrew Ng’s lecture Series- Stanford Fall 2018

This a very classic material used by many people in the industry even with a very rigorous background in ML and computer science. I will not attempt to start describing how clear and crisp Prof Ng’s lectures are. This lecture series will help the reader speak the language of the computer science domain if the reader is from a different domain like Economics, Anthropology or Psychology.

I often times faced a challenge in my early interview days, when I realized that economists and other social scientists have a very good understanding of how a Linear model works, but they are not familiar with computer science terms like “hypothesis” and “cost function” and “loss function”. Tech industry is populated by computer science graduates and they seldom bother to find out how these concepts are referred in other domains. One can easily get rejected by interviewers if one is not able to explain oneself using the terminologies used by the computer scientists. The first few lectures of this course is a very good ground to inculcate all that.

This lecture series also empowers one with the necessary mathematical equations and the distributional assumptions that are the key areas often tested in ML depth interviews. One needs to be able to express the statistical assumptions behind a Linear model and derive the normal equations using calculus, if asked in an interview.

d. Andrew Ng’s Course on Deep Learning in CourseEra

This is another classic material used by almost everybody in the industry. This course is divided into five sub-courses, each one of them being profusely used by people in many spheres of their work. The availability of this course have democratized education, where high quality materials, that were previously accessible to a select few students fortunate enough to get admission in the Stanford University, are now available to anyone who is interested in learning. Deep Learning is no longer a mystery word, understood by a select few of our elite colleagues.

The course starts with Prof Ng’s explanation of how a simple logistic Regression model can be explained by a simple Neural Network (NN) having a single layer and then he slowly builds on that concept and converts a simple NN having one layer to a complex NN with many nodes and hidden layers. The course provides the students with both the mathematical formulations and the python code required to implement these algorithms from scratch.

Recent ML depth interviews have become harder as the candidates are asked to generate the code of an ML algorithm from scratch using gradient descent. Linear/Logistic regression and KNN are popular algorithms in this category. Prof Ng’s course in deep learning is crucial in helping a candidate learn to implement these algorithms using simple numpy arrays only.

The third and the fourth courses are short courses where Prof Ng teaches candidates the essential tools required to implement NN in practice.

The fourth and the fifth course in this series involve Convolutional Neural network (CNN) and Recurrent Neural Network (RNN)- specialized courses for the fields of Computer Vision (CV)and Natural Language Processing (NLP). The fourth and the fifth course are independent of each other and one does not need to take them in any specific order.

Hope this article will be helpful in providing the reader with a concrete study plan for the second stage of the interview process for Data Science and MLE positions. Personally, I found this to be an enjoyable and a rewarding journey where I learnt a ton of new things. It’s much less daunting when compared to the coding interviews which requires an individual to be machine in order to be successful. Good luck and all the best!