👩🏻💻 Miki’s 🔥Hot-Takes🔥 on MLE Interviews: Types of Roles & Interview Prep (Part 1/2)
The first part of a loquacious guide to resources, tips, & lessons learned interviewing for Machine Learning Engineer roles.
My goals for this blog series are to demystify the interview process for Machine Learning Engineer roles and avail readers of my personal experiences as a useful guide post.
Over a period of four months I interviewed at multiple companies for MLE roles ranging from associate to senior (beyond the initial recruiter screen).
In the beginning of my search I was confused, frustrated, and at times felt the process of getting an MLE offer was even more opaque than getting an offer as a data scientist.
My goal is to provide some insight into the types of roles available in MLOps+Eng, resources I found useful (depending on whether I was studying or interviewing), and some of the advice that helped me secure three offers.
This blog post is for people who are:
- Actively interviewing for MLE roles and trying to prioritize their preparation;
- Thinking of making a switch from data science and analytics to ML ops & engineering and would like to understand the different opportunities within MLOps & ML engineering;
- Interested in learning more about MLOps & ML Engineering and want some recommendations for awesome educational content.
Warning: This is an opinionated post from someone talking about their particular experience. Results not guaranteed.
Main Sections
In Part 1 we’ll cover the sections marked with 👉🏻 (everything else is in Part 2):
- 👉🏻 [Section 1] Types of roles in ML Engineering, specifically the difference between Research vs Ops (Sherlock Holmes or Bob the Builder — Pick One At Least For Now);
- 👉🏻 [Section 2A] ⛔️ Types of interviews, specifically FAANG-style vs holistic-style interviews (Leetcode Queen, Holistic, or Blended); (⛔️ — Not Writing About For Now, may revisit in a separate post)
- 👉🏻 [Section 2B] How to deal with the Data Structures & Algorithms portion of interviews (Leetcode Queen, Holistic, or Blended)
- [Section 3] Top 3 pieces of advice for the MLE job hunt (Advice for the MLE Job Search — Optimize the Bottleneck, 80/20 Your Efforts, Ask for Help);
- [Section 4] Top 4 general pieces of advice (Advice from the Soapbox — Cash is King, Get a Squad, Creature Comforts, Be High-Value);
- [Section 5] My favorite resources I used (The Good Stuff — Excellent Victuals for the Discerning Scholar’s Palate).
Resource Legend
☝🏻 Quick note: Throughout the post I’ll link supplementary resources in addition to the resource list at the bottom. In order to help you prioritize them, I’ve added a little guide via emojis next to each link.
And to be frank, sometimes other people do a way better job explaining certain concepts. Why reinvent the wheel when I can just guide you to their doorstep?
Section 1: Sherlock Holmes 🕵🏻♀️ or Bob the Builder 👷🏻♀️— Pick One (At Least For Now)
🤔 Understand the options
If you’re trying to figure out the right jobs to apply for, a crucial first step is to ignore the titles, pick the type of work you want to be doing, and focus on corresponding job listings.
Being intentional and directed on the type of work will massively cut down on the wasted effort of pursuing interviews for roles that are fundamentally misaligned with your goals and will help you target the 80/20 skills and experiences you need to develop to secure those offers.
I find it helpful to think of roles in analytics, data science, and machine learning as being organized fundamentally along two axes:
- Axis 1 (Type of Work): Strategy vs Research vs Engineering + Ops
- Axis 2 (Output): Analysis vs Data vs ML Product
Having a clear target and framework will help you position yourself as a candidate by structuring your past accomplishments and identifying gaps between where you are now and where you need to be.
For example: Earlier in my career I produced analytics focused on delivering strategic insights and recommendations using a combination of analytical tools (like R, Python, SQL) and reports & dashboards (created using Tableau, PowerBI, etc), all powered by data from internal business applications (like Salesforce, Marketo, etc).
I then moved on to data science where I was using more powerful tools to deliver predictions. And now I work as an ML Engineer partnering with the Data Science organization, focused on developing tools, infrastructure and processes to help power innovation at scale for Mailchimp 💌🐒.
Being able to zero-in on the types of problems I want to be solving and connecting my past projects and experiences to what the business and teams need also allowed me to structure my narrative to clearly illustrate why I decided to move from data science to engineering and Ops.
When working as a data scientist, the models I developed were coded locally, in isolation, and weren’t always used in production (depending on whether there was infrastructure or support for deployment and monitoring).
In struggling deeply with questions around how to make the models I developed useful, resilient, scalable, and testable I realized that developing “special knowledge” in bringing machine learning to production was an area of opportunity.
How many times was I asked the sweat-inducing question of “Great, now how do we automate this? What happens if you go on vacation? How do you know the model will work? What happens when it stops working? What’s the likelihood that the model stops working, you’re not around and you still get to keep your job?”
And how many others are asked these same questions?
As I looked up and all around, I saw my most successful teammates either performing interesting, envelope pushing research or developing the systems that enabled the research to go somewhere.
👈🏻 👉🏻 Choose between producing Research or Software
Like many machine learning engineers (and data engineers) at some point I recognized that I cared more about the question “How do I make data science & machine learning useful?” as opposed to “How do I push the envelope on domain specific DS/ML questions?”.
There’s a creative tension between research and software development, as they’re both really, really hard disciplines that are integral to the development of powerful and cool ML systems.
Research alone is insufficient for the development of production ML.
Well-designed machine learning systems need to check a ton of boxes and are composed of code and tools beyond ML.
The teams that typically support an ML system are composed of an equally varied group of individuals with complementary skills sets as opposed to hero engineers or data scientists.
Because as you can see in the diagram below (linked from Google’s documentation “MLOps: Continuous delivery and automation pipelines in machine learning”), there is a lot of ground for just a single data scientist to cover.
As Chip Huyen commented in her talk with the Stanford MLSys Seminar Series, “Principles of Good Machine Learning Systems Design”,
“Deploying is Easy. Deploying reliably is hard.”
And as she further pointed out in the same talk, the aims of ML in research and ML in development are quite different.
So given all the necessary requirements of a functional, scalable delivery mechanism for many models, one of the biggest favors you could do for yourself is to pick a path.
Do you produce research or software?
That path doesn’t need to be the only path you travel in your career (and picking a path doesn’t preclude you from other paths down the road). But your life will be made much easier if you’re laser focused on solving specific and related problems.
🙅🏻♀ ️Can I Do Both?
Yes — but do you really want to at the same time? More importantly, can you do both and still be happy and successful?
One of my favorite and most influential mentors-at-a-distance describes advice that Warren Buffet supposedly gave an employee about prioritization.
As the story goes, Buffet asked his pilot what his top 25 goals were. Out of those 25 listed, the pilot needed to pick just 5 to focus on. After the pilot picked his top 5, Buffet asked what the pilot would do with the next 20 goals, to which the pilot replied he would work on them intermittently.
Not a bad idea, right?
WRONG! — At least according to Warren Buffet who replied:
Bad Steve. Very Bad Steve for trying to multitask and accomplish too many goals at once thereby decreasing momentum through context switching.
Research and engineering require a different set of skills and expertise and while there’s some crossover, according to people like Assaf Pinhasi (in “Machine Learning products are built by teams, not unicorns”), Semi Koen (in “Being a Data Scientist does not make you a Software Engineer!”), and Shreya Shankar (in “Reflecting on a year of making machine learning actually useful”) there is definitely a not-insignificant gap to cross in order to succeed in both domains (not impossible but definitely significant .
While the ability to succeed in both research and engineering is a worthy aspiration, when domains are different enough, Anders Ericsson points out that success in one domain doesn’t guarantee success in another.
Specifically he writes in Peak: Secrets from the New Science of Expertise:
“A key fact about such mental representations is that they are very ‘domain specific,’ that is, they apply only to the skill for which they were developed. We saw this with Steve Faloon: the mental representations he had devised to remember strings of digits did nothing to improve his memory for strings of letters. Similarly, a chess player’s mental representations will give him or her no advantage over others in tests involving general visuospatial abilities, and a diver’s mental representations will be useless for basketball. This explains a crucial fact about expert performance in general: there is no such thing as developing a general skill. You don’t train your memory; you train your memory for strings of digits or for collections of words or for people’s faces. You don’t train to become an athlete; you train to become a gymnast or a sprinter or a marathoner”
Okay I’ve bought in & chosen my focus — Now what? 🤷🏻♀️
Hopefully I’ve convinced you to consider targeting a particular role persona (or type of work) in your quest to getting a job in Machine Learning. Roles are either oriented towards research (with titles like “ML Researcher”, etc) or towards software development or ops (with titles like “Machine Learning Engineer”, etc).
Sometimes you’ll see titles like “Applied ML Researcher” which typically means an individual focused on research that can code (which can include implementing algorithms from research papers and finding ways to use them in the product to make money).
For my most recent job search my target was specifically Machine Learning Engineer roles with a focus on MLOps (⚠️ and hopefully you’ve decided that’s your focus too or else the rest of this blog post is basically useless ⚠️). My friend Vin Vashishta talks more about this type of role in “What is MLOps and What Do You Need To Know To Become An ML Engineer?”.
In the following sections, I’ll dive into my experiences interviewing for MLE+MLOps roles (as opposed to research ML roles or data scientist roles) but feel free to check out some of the other perspectives linked below regardless (especially if you’re trying to decide your next move).
📝 Additional Noteworthy Writings About MLOps, MLEng, Positioning Yourself as a Candidate, & Career Crafting
- Eugen Yan’s blog posts are truly enjoyable reads and are published at uncannily timely moments. Three specific blog posts worth highlighting and particularly relevant to the questions of “What is an ML Engineer?’ and “To research or develop?” are “Applied / Research Scientist, ML Engineer: What’s the Difference?”, “The Metagame of Applying Machine Learning”, and “A Practical Guide to Maintaining Machine Learning in Production”.
- Understand value-creation and value-capture through the words of Naval Ravikant via his admonishments to “FIND AND BUILD SPECIFIC KNOWLEDGE”, “PRIORITIZE AND FOCUS”, and to “FIND A POSITION OF LEVERAGE”.
- Reading Omar Itani’s blog post “Think Like a Craftsman and Master a Skillset; it Will Give You Meaning and Transform Your Career” is a great reminder that career growth and satisfaction is the result of intentional and incremental growth in valuable skills. Cal Newport’s “The Career Craftsman Manifesto” is a celebration of the sentiment that “the color of your parachute is irrelevant if you take the time to get good at flying the damn plane in the first place.”
- One of the best written pieces I’ve come across so far on MLOps, “MartinFowler.com: Continuous Delivery for Machine Learning” is detailed & through must-read.
Section 2A: Leetcode Queen, Holistic, or Blended (⛔️ NOTE: This section is UNDER CONSTRUCTION. Trying to figure out if I really want to relive interview memories.)
To be honest, I almost don’t want to talk about the interviews, mostly because the Leetcode based interviews were incredibly painful and took a lot of time to prepare for (and eventually I stopped accepting interviews where Leetcode was a major component). 🤷🏻♀️Nevertheless, here are my 🔥hot-takes and recommendations.
Style #1: The FAANG Way — Because the goal is minimizing False Positives. 🙅🏻♀️
⛔️ This section under construction.
Style #2: The Holistic Way — Because not every company is a FAANG company & candidates are people too. 🙆🏻♀️
⛔️ This section under construction.
Style #3: The Blended Way — Because ML Eng & Ops is hard, requires multiple skills, & is a practice, not a job title but we still need a structure to match candidates.🧏🏻♀️
⛔️ This section under construction.
Leaving Leetcode land 👋🏻
After a certain point where I started generating enough interview opportunities where the focus wasn’t on LeetCode interviews, I began to prioritize ML system design based interviews and moved away from pursuing LeetCode based interviews.
When I first started applying for ML Eng roles, a majority of my interviews followed the Leetcode/FAANG style. Once I started getting into the swing of interviewing and I had a more robust pipeline of interviews with a more diverse mix of companies, the percentage of my interviews involving LeetCode moved from 80% to 20%.
And while total compensation wasn’t always as high as what I would have gotten at a FAANG company, the base salaries for the three and half offers I did get were within $10K of the closest level and at times more than made up for the lack of equity through guaranteed increased 401K matches, bonuses, amazing company culture and mentoring, and opportunities for promotion.
With that being said, here is my best advice for…
Section 2B: 👊 Breaking through the Data Structures & Algorithms interview
My advice on studying data structures & algorithms, especially if you’re informally taught?
Before signing up for a 3–6 month data structures & algorithms class and mindlessly solving LeetCode problems, I would first:
- Understand what the expectation is for a technical interview. Watch this video where Gayle Laakmann McDowell of Cracking the Coding Interview fame does walkthrough of the Facebook technical interview.
- Make a list of the data structures & algorithms to study. Either your recruiter will provide a list or you can create one from the chapters of the EPI book (linked below) or Cracking the Coding Interview book (also linked below).
- Check out Bukola’s suggestions on how to take notes for DS&A study in How To Master Data Structures & Algorithms (Study Strategies), especially if you’re a huge Notion.so fan. She breaks down exactly the type of information you should know about the data structure or algorithm you’re studying.
- Start learning the data structures & algorithms one by one — specifically the type of details and information Bukola talks about in her note-taking video.
As you’re studying each data structure and algorithm I would:
👉🏻 Step 1: Learn the algorithm as an abstract data structure ➡️
👉🏻 Step 2: Understand the Python (or relevant language) implementation ➡️
👉🏻 Step 3: Practice some corresponding problems in LeetCode ➡️
👉🏻 Step 4: Use the feedback on your solution to refine your understanding of how to optimize your solution (from a space & time complexity standpoint) ➡️
👉🏻Step 5: Then focus on code style.
Example: If you’re starting with arrays, learn about arrays as an abstract data structure, then learn the Python (or relevant language) implementation of arrays, analyze sample array problems to deepen your understanding of the underlying principles and time-space complexity, then practice array problems on LeetCode (using the feedback to further refine your problem solving skills using array data structures).
Once you’ve successfully solved some medium-hard array problems, then move on to linked lists and repeat the process.
💡 Additional recommendations on DS&A studying
- For the simplest ELI5 explanations I’d start with Hackerrank’s Data Structures Youtube playlist where each video is less than 10 min long, narrated by Gayle, and focuses on a single data structure or algorithm. If you need a deeper understanding of the “Whys” (like “Why does it matter if a tree is balanced?”, “Why choose sorting algorithm X over Y?”, etc) the Simple Snippets Playlist: Data Structures & Algorithms in C++ Playlist is excellent. If you need additional explanations on algorithms (with visual imagery) Grokking Algorithms is potentially a useful resource (although at times I felt the metaphors confused my understanding more so than the actual technical explanations).
- If you prefer written explanations over video (or want to supplement) read the explanation/description of the DS/A in Cracking the Coding Interview (without solving the problems), read the corresponding chapter in Elements of Programming Interviews in Python for the Python implementation and language details, and implement the examples in EPI-Python in the LeetCode playground or in your IDE/scripting tool of choice.
- Once you have a conceptual understanding of DS/A, you understand the “why” and “how” of usage and implementation in your chosen language, you’ve analyzed some example problems, that’s when it makes sense to use LeetCode in a targeted manner. I’ve seen too many posts from people who went straight to LeetCode, didn’t understand the problem they were solving, the tools they had available, and then burned out because they kept failing to get better and didn’t understand why.
- Finally once you’ve completed some practice problems, it makes sense to improve your interviewing code style and one of the best resources I’ve used is the Real Python Learning Path: Ace Your Python Coding Interview — Specifically the video courses Python Coding Interviews: Tips & Best Practices and Introduction to Sorting Algorithms in Python were fantastic reviews of problem solving leveraging Pythonic code.
📝 Additional Resources About Interviewing for ML Ops/ML Engineering Roles
- Patrick Halina’s ML Eng Interview Guide Series — Halina is a Staff ML Engineer at Pinterest who’s writing a blog post series on the Systems Design portion of interviews. I’d highly recommend checking out Part 2, the “ML Systems Design Interview Guide”, for additional information about what to expect in ML Eng interviews.
- Rahul Agarwal wrote a detailed blog post describing his experience interviewing as an MLE at Facebook. Anjul Tyagi talks about his experience from the perspective of interviewing for the PhD internship. Facebook and similar companies are upfront about their interview process and can be really generous about scheduling. While there’s a lot less guesswork about what’s covered, the bar is higher on testing for CS fundamentals and screening even potentially viable candidates early in the process.
- For additional perspectives on interview for Machine Learning Engineer interviews, Machine Learning Interview: Crack the ML interview has some interesting perspectives although important head’s up, the interviews depicted skew heavily towards FAANG companies (which heavily bias towards a very specific kind of intense, multi-round, Leetcode and whiteboarding focused interviews) and also bias towards more ML research oriented or applied ML research oriented roles (with very few MLOps/Eng interviews represented in the mix). Personally more than half my interview for ML Eng roles required no Leetcode or algorithm whiteboarding sessions so be warned, the interviews you’ll read about aren’t anywhere near representative for the market.
Coming Up Next in Part 2
In Part 1 we covered the sections marked with 👉🏻.
In Part 2 we’ll cover everything else marked with 🎉.
- 👉🏻 [Section 1] Types of roles in ML Engineering, specifically the difference between Research vs Ops (Sherlock Holmes or Bob the Builder — Pick One At Least For Now);
- 👉🏻 [Section 2A] ⛔️ Types of interviews, specifically FAANG-style vs holistic-style interviews (Leetcode Queen, Holistic, or Blended); (⛔️ — Not Writing About For Now, may revisit in a separate post)
- 👉🏻 [Section 2B] How to deal with the Data Structures & Algorithms portion of interviews (Leetcode Queen, Holistic, or Blended)
- 🎉 [Section 3] Top 3 pieces of advice for the MLE job hunt (Advice for the MLE Job Search — Optimize the Bottleneck, 80/20 Your Efforts, Ask for Help);
- 🎉 [Section 4] Top 4 general pieces of advice (Advice from the Soapbox — Cash is King, Get a Squad, Creature Comforts, Be High-Value);
- 🎉 [Section 5] My favorite resources I used (The Good Stuff — Excellent Victuals for the Discerning Scholar’s Palate).
More About Me
Interested in following along with my journey? One of the easiest ways is by connecting through LinkedIn or GitHub. Feel free to send me an invite and let me know what you thought about this post!
I’ve also written about my journey to becoming a data scientist in a series of posts starting here: ✂️Breaking into Data Science- From Hair Salon to Data Scientist (Ch.1)🔍
And if you really love what I wrote, consider buying me a coffee at www.buymeacoffee.com/mmbazel & keep me writing! ☕
