A Different Take on AWS Machine Learning Specialty Certification
Before I got certified, I didn’t know where to start; here’s a quick guide.
This year has seen the definitive launch of the AWS Machine Learning Specialty. You may have heard horror stories about how this is the hardest of all AWS certifications. I wouldn’t know about that, to be honest, but what’s clear is that it is the most unique of them all.
The reason is that this specialty doesn’t rely on AWS services knowledge alone, but expects you to be comfortable about machine learning and deep learning in a wide sense. If you’ve been working as a data scientist for the past years, using models or have an academic mathematical/statistical background, you should be Ok with that part. That wasn’t exactly my case. AWS emphatically recommends to have at least an AWS Associate Certification and two years of cloud technology experience before going for this one. When I took (and passed) the exam, I had bit less than two years of experience as data scientist and just the AWS Cloud Practitioner Certification. How did I face it, then?
The beginning: Exam Preparation
All free training courses (called ‘paths’) provided by AWS here rely heavily on video tutorials done by different experts and/or product owners, and as such, they tend to be very irregular in their quality. Most of them is just AWS internal content made available to everyone, which may give you a good starting point if you know nothing about the subject (from mathematical grounds of machine learning to architectural solutions in AWS), but won’t be substitute of a decent background in the same areas. My recommendation: read the official white papers from Amazon to get an idea of all the products in their cloud, with special emphasis in Sagemaker.
If you’re one of those students that prepare exams by overfitting (I mean, trying to memorise all areas of the exam but failing to understand them), I must tell you; it’s not the best approach. The best scenario for taking the AWS Machine Learning Certification is understanding how AWS services interact with each other, how they work internally and what possibilities they have to offer.
A few paid courses are available to prepare the exam; Linux Academy’s was still a work in progress in the second half of 2019, and A Cloud Guru is probably the best in terms of quality and syllabus. Exam simulators in both courses are way easier than the real thing, specially the one Linux Academy has. The best that can be said about them is that it’s definitely good to do simulations with the same conditions as the real exam. 180 minutes can be hard to manage if you’ve never done this type of exams and/or, as many people, just hate taking them.
Content Outline
Amazon provides a list of domains you’ll need to know, from Data Engineering and ML Implementations and Operations (each representing 20% of the exam), Exploratory Data Analysis (24%) and Modelling (36%). Some areas intersect with others in a way that may be hard to make sense of, so my view is that a simpler, more real partition would be like this:
Machine Learning and Deep Learning (50% of the exam)
You should be comfortable with the whole ML cycle; from data collection and data preparation to exploratory data analysis and modelling. A good grasp on how to formulate problems and measure success is paramount. Which metrics to use play an important role in the exam. What types of data (structured, unstructured) exist and what makes good data in a machine learning project.
Data preparation is a key step in data science, and you should expect a few questions on missing value handling, categorical encoding, imputation and other feature engineering steps. Pay attention to scaling (normalization and standarization) for numerical data, n-grams and bag-of-words for text data, as well as Term Frequency - Inverse Document Frequency (tf-idf), as you may have to solve simple corpus data problems. Also, keep in mind how and why recordIO protobuf format will work better in AWS jobs.
Some questions may present you visualizations of data and ask you something about that data. This is clearly an attempt to test your skills in data analysis. You should be comfortable knowing the different types of graphs and their aim (comparison, composition, relationships and distributions). Be aware of how they are used and how can they help in exploratory data analysis.
Model design is obviously present in the exam too. How to select a good model, what ML approach is better in a situation (regressions, classifications) and what metrics and strategies should be used. About the algorithms, you should be comfortable with all the classics (K-Means and its differences with K-Nearest Neighbours, Random Forests and Decision Trees) as well as with Convolutional Neural Networks.
You’ll need to understand the concepts of training, test and validation data, identify potential biases introduced by insufficient splits, and additional measures that could be used to increase data value. What kind of generalization are we looking for in a ML process and how it will be consumed (real time, batch, API applications), and how to tell if the generalization is working (accuracy tests) is an all-present topic throughout the exam. You’ll need to be able to explain how Confusion Matrices, Recall, Precision and False Positive Rate work.
About evaluating models, you’ll need to understand offline and online validation, and at least conceptually know about canary deployments. Underfitting and overfitting and how to overcome them, regression accuracy (RMSE), histograms and skewedness in them, AUC metrics in classification, and all the trade-offs in evaluation that might call for different optimizations are all very important in the exam. Also, model tuning specifics and at least some grasp in Bayesian optimization will help a lot.
Sagemaker (25% of the exam)
While knowledge of Machine Learning is expected from you in the exam, the star of the game is Sagemaker, AWS’ central and pivotal service for all things AI. Sagemaker helps you through the whole ML cycle from start to finish, and you’ll probably have to explain how AWS services interact with it. Keep in mind that it will appear in questions quite often, and you need to know its intricacies when it comes to training models (how to create a training job API, how to set up an Elastic Container Repository, inferences).
Another important thing to know are Sagemaker built-in algorithms. From Linear Learners to Factorization Machines, Image Analysis and Anomaly Detection, and differences amongst text analysis algorithms (managing and explaining when to use LDA, Neural Topic Modeling, Seq2Seq and Blazing Text can and probably will appear often). As a widely used algorithm both in regression and classification problems, XGBoost shows up too, as in comparisons with other algorithms. Keep in mind that knowing the main built-in algorithms hyperparameters and metrics is a must.
Last but not least, knowing how to take Sagemaker models to production and how Jupyter notebooks work is needed. Sagemaker hosting services (how to create endpoint configurations) and inference pipelines to chain together algorithms, docker containers and elastic inference can pop up in the questions. You should also know how automatic scaling works.
Other AWS services (25% of the exam)
Here’s where the exam gets similar to other AWS certifications. If you already have a solutions architect or big data certification (I didn’t), all this will surely be familiar. The most important services here are AWS Glue and Athena (for ETL jobs), the Kinesis family (for streaming data), S3, RDS, DynamoDB and Redshift (as data stores) and the Hadooq cluster ecosystem Elastic Map Reduce (EMR). You should understand in depth how they all work, what are their strengths and why should be used in specific situations. The white papers for this services are quite good, but ideally you should have some experience working with them. If that’s your case, you are covered.
Some questions may make you choose the correct order of services to use in a given business situation. You will need to know each service, its purpose and use-cases that will be presented to you in the questions. Keep in mind all monitoring and evaluation tools for deployments AWS has to offer. I’d recommend to make an effort in understanding CloudWatch and CloudTrail in detail if you’ve never used them before.
There’s a few AI developer services that you must know, but the good news is that they’re quite simple: Forecast, Lex, Personalize, Polly, Rekognition, Transcribe and Translate are all easy-to-use services available in the console that you just need to try yourself to understand.
For deployments outside Sagemaker, AWS offers some things you should check out: Elastic Container Service, EC2 AMIs, Elastic Map Reduce and on-premises options (MXNet and TensorFlow frameworks).
Managing your time in the exam
While you’ll have a bit less than three minutes per question (which is more than enough), I find the best strategy is to answer first all those questions you know, -or have little doubt about- and leave the harder ones for later.
Some questions are quite verbose, but after reading them, you’ll see they may just be trying to confuse you. This is an important point: the exam will try to trick you. It’ll try to find where your knowledge is thinner with very similar answers that may superficially look the same. If a question seems complicated, leave it for later. The worst thing you can do is to get stuck in a question and spend 10 minutes there, because you’ll probably need them at the end of the test.
Let’s wrap it up:
If you got here you may be a bit overwhelmed by the information, so here’s my final five tips to facing the exam:
1- Read the whitepapers, specially Sagemaker’s, but ideally all those from the main AWS services, as stated above.
2- Try as many AWS services as you can if you haven’t worked with them in the past. Play with them, feel comfortable.
3- Forget AWS official content outline percentages; keep in mind it’s 50% ML/DL, 25% Sagemaker and 25% other AWS services.
4- Don’t get stuck and/or think about how much time you have left; 180 minutes is enough, but use that time wisely.
5- If questions are verbose at first sight, read the answers carefully. Even before the questions themselves. They usually point to the service or solution better than the question.
Bonus tip: Bring or ask for a pen in your examination center! You may need to solve (simple) problems.
Good luck!