A Software Engineer’s Guide to AI and Machine Learning
From deep learning to our own critical thinking, intelligence is about interacting with the world and building mental models. These models then serve to empower the user and lead them toward informed decisions.
I want to introduce a couple of my own mental models that I’ve put together during my deep dives into the world of machine learning. I was inspired by this article on a VC’s guide to ML and figured I’d provide my perspective as a software engineer who ultimately wants to build an ML powered product. Whether you’re just beginning your career or well into your profession looking to found a startup, I hope I can provide some insights into what ML is, what it can do for you, and how you can get started!
ML as the new “Software”
Marc Andreessen famously said “Software is eating the world.” I believe machine learning is the next “Software” that will revolutionize the coming decade. Advancements in ML will unlock tremendous value in a plethora of industries. These range from fraud & risk detection to cell protein structure prediction.
Machine learning is even changing how we think about programming. Traditionally we write software that solves a job to be done. Machine learning flips this on its head. How can we set up a bunch of rules so that a machine can follow these rules and make informed decisions? How can these informed decisions lead us to solving a job that couldn’t have been solved with just software?
Takeaways:
- ML/AI will revolutionize the coming decade so invest in your understanding!
ML as intelligent representation
Understanding the correct representation is like understanding the rules to a game. You can’t operate effectively if you don’t know the rules! This is relevant in all walks of life. Copernicus pushed forward our understanding of the solar system with his improved Heliocentric model. Marie Curie furthered our knowledge of atomic theory by showing us the atom is actually divisible and made of subatomic particles. Charlie Munger taught us better investing practices with his concept of using a latticework of mental models to make smart choices.
Likewise in machine learning, we can better understand our problems and improve our algorithms if we intelligently pick how we represent our data. Good representation makes it easier to “teach” our algorithms. One good example is representing the text in a word document. A simple approach would be to represent it as a bag of words, where we count the occurrence of each word. The flaw to this approach is that we don’t capture any of the relationships between words nor do we consider weighing words based on how frequently they appear in other documents. Enter TF-IDF and word2vec. These new methods for representation have drastically improved the effectiveness of our ML models.
Takeaways:
- always seek to understand and question our mental models of both the problem and solution
ML as data
There can be no machine learning without data. There can be no wisdom without life experience. Computer algorithms and humans alike need data to learn! As such, when embarking on an ML project, be prepared to go looking for data. This could entail searching for existing datasets on Google, Kaggle, or some other dataset repository. This could also entail creating your own dataset in the wild. For one of my projects to create a robust spam classifier, I downloaded and stored real phishing emails from my own Gmail inbox for later use. For training GPT-3, OpenAI relied on the Common Crawl corpus which contains petabytes of real web data scraped and archived.
As you dive deeper into the world of big data, there will be other tools to help you gather, store, and query your data. Be sure to get familiar with technologies such as Apache Spark, Flink, and Airflow. They’ll help you set up data processing at scale.
Takeaways:
- When starting any ML project, think about where and how you’ll get your data.
- Let your programming skills shine, from writing scripts that scrape data to setting up an ETL/ELT pipeline to prepare data.
- If you plan on being an ML product founder, be familiar with the latest tools of the industry.
ML as a system
ML systems are no different from software systems. Both need metrics and monitoring, some database or data lake, and servers to deploy code and models. Unfortunately, some of the same sins of traditional software also plague ML systems such as tech debt and monolithic codebases.
My advice would be to think holistically about how you’re going to build your product. What are all the moving pieces? As someone who wishes to someday build a product powered by ML, I try to consider what it would take to “productionize” a machine learning model. For example, how can we make a program that can classify something as hotdog vs not hotdog useable for the end user? Will you store all the data in an AWS S3 bucket? Will there be servers serving the ML model for hotdog prediction?
Takeaways:
- When designing an ML product, think about how to productionize the algorithm and the underlying system architecture — it’s like architecting traditional software systems!
ML as just another skillset to learn
Lastly I’d like to leave you with the general guidance of experimenting often! Machine learning is conducted with various computing tools. Learning the ins and outs of Tensorflow is a lot like trying out a new frontend framework. Python can feel familiar if you’re already a Ruby or JS expert.
The ML world can seem daunting but have faith. Get your hands dirty with tutorials, continually take notes, and build out diagrams like this for understanding!
The level of technical ability you need to show is not lowered, it’s even higher when you don’t have the educational background, but it’s totally possible.
— Dario Amodei, Researcher at OpenAI, on entering the field without a doctorate in machine learning
Takeaways:
- Machine learning is a lot like programming; you get better by implementing ML projects over and over.
- We all want to work on the most important problems in our field. If you decide learning ML is the key to tackling some of these problems, make sure you have laser focus on the next steps while maintaining your long term vision. First I want to learn about logistical regression. Next I want to… and so on. Remember to take notes!
Resources section
I hope this article and the following resources help in getting started!
Great books to get started
Online courses
Aaaaaand here’s a bunch of ML concepts as animal gifs. Also here’s a more technical overview of ML in a nutshell I wrote a while back. Godspeed on your ML journey and thanks for reading!
