How Much ML Should Engineers in Tech Really Know?
As people working in the ML field, we often talk about how much engineering skills MLOps/ML engineers should have. This time, let’s talk about how much other engineers should know about ML. (inspired from this article and my current team at Booking).
As an MLOps engineer, I have worked with different professionals — software engineers, DevOps engineers, and platform engineers. All of these roles are important for successful deployment and operationalization of ML models. Close collaboration is inevitable, the question then to ask is how much non-ML engineers should know about machine learning. Whether they are writing code, managing infrastructure, or provisioning resources, understanding a bit of ML wouldn’t hurt.
Disclaimer: We are primarily focusing on non-ML roles involved in ML product development. Each organization has a unique setup, sometimes placing software engineers in ML engineer roles or DevOps engineers in MLOps positions. Not all companies have dedicated ML or MLOps engineers. Assigning other roles to help data scientists in model deployment can be a great solution if these individuals have some understanding of ML.
From our (ML folks) perspective, what we are doing in the ML field might seem like a black box, or like a traditional software with additional complexity. I used to assume that everyone in the tech field had at least a basic understanding of ML. But, that’s not always the case. The level of ML knowledge can vary among engineers.
The core
At a minimum, all engineers involved in ML products should have a basic understanding of what machine learning is and how it differs from traditional software systems. ML models, unlike typical software programs, are not hard-coded with explicit instructions. Instead, they learn from data, which changes the development, deployment, and maintenance processes.
This basic knowledge helps everyone involved understand the specific requirements and limitations that come with integrating ML models into production environments.
Drawing the line
A software engineer who helps to package and deploy a model should know:
- Model APIs: How to work with a model using an API, e.g. input and output formats, error handling, and performance considerations.
- Re-Training and Versioning: Code versioning and model versioning are different. Each time a model is re-trained with updated datasets, a new model artifact is created, which can be deployed as a new update.
- Monitoring and Logging: Important to monitor the model performance in addition to the service metrics
A DevOps Engineer who helps create pipelines:
- CI/CD for ML: Understanding how to integrate ML models into continuous integration and deployment pipelines. This includes model versioning, automated testing, and rollback procedures.
- Containerization and Orchestration: Containerizing ML models, typically by including their model artifacts. Orchestrating their deployment using tools like Airflow. Understanding resource requirements for CPU, memory for training step, and GPUs or TPUs.
- Security considerations: Being aware of sensitive data access by the model or service, and ensuring secure model deployment.
A Platform Engineers who creates resources for ML products:
- Resource provisioning: Understanding the resource requirements for training and deploying ML models, which might include specialized hardware like GPUs or TPUs.
In a nutshell
Software engineers should understand the importance of handling model APIs, code and model versioning, and ensuring both model performance and service metrics are monitored effectively.
DevOps engineers should focus on integrating ML models into CI/CD pipelines, managing containerization and orchestration, and securing the deployment process.
Platform engineers should understand why specialized hardware is required, and ensure that the infrastructure can scale to meet the demands of ML models.
Not everyone playing a role in end-to-end ML product development needs to become an ML expert. Some fundamental knowledge enables each player to collaborate effectively with ML teams and contribute to the successful deployment.
Some resources to learn fundamental ML knowledge
Applied Machine Learning and AI for Engineers by Jeff Prosise (book)
