Deploying your own machine learning model with Amazon Sagemaker
In this article, we will go through the following points:
- Introduction
- Deploying a model in practice
- Have a little fun (just some tips)
- Conclusion
Some code examples that are in this article are available at gist.
1. Introduction
Amazon Sagemaker is a service that makes it easy to create quickly, train, and implement machine learning (ML) models with the set of available solutions. It is a helpful tool for data scientists to start a project without concern about infrastructure. AWS indicates on its website a MLOps Framework reference architecture [Fig1] — where you can see Amazon Sagemaker being one of the services.
Understanding how each component connects is advantageous, but sometimes it can be challenging to know how they fit together. And it is good to be aware that this service is not free. You can take a look at the prices at this link.
1.1 Motivation
Here in Sinch, we would like to understand our customers better, and one idea was to apply a topic modeling solution to discover their preferences. We also can use AWS as our cloud environment to develop machine learning solutions. Besides that, it has services that focus on text solutions, such as Amazon Comprehend and Amazon Sagemaker, with prebuilt models, i.e., NTM (Neural Topic Modeling) and LDA (Latent Dirichlet Allocation).
After test those available services, we decided to create our topic model solution to get better results because of the peculiarity of our clients. But, we need to create an automatic pipeline to deploy this model, and Amazon Sagemaker combined with other Amazon services can be an option.
This article aims to show how to deploy our own topic model solution using those Amazon tools.
We will use almost every component of the framework and a train.py file with our solution of topic modeling. This solution uses some standard libraries as Gensin and Sklearn to test NMF, LDA, and LSA models [3,4]. We will use the OCTIS library [5,6], which provides the following performance metrics to evaluate the topic created:
- Topic coherence
- Topic significance metrics
- Diversity metrics
- Classification metrics
2. Deploying a model in practice
We can simplify the process into three main steps: model Build, model Train, and model Deploy.
2.1 Model Build
First, it will be necessary to create a docker image to be pushed inside amazon ECR (Amazon Elastic Container Registry). You can do it in any environment of your preference. In our case, we take advantage of the jupyter notebook instances from Amazon Sagemaker. Here is an example of the Amazon Sagemaker console page:
The main advantage is that you can easily set up a machine with the size of your choice and the environment preconfigured. And, if do you have any code inside git, you can connect with it.
Inside this jupyter notebook instance, we will create all the requirements files to create the docker file:
- Train script file
It is the training script that we make to run the topic modeling solution. It should have the train function and the optimization function. Also, the script train should save the final model inside an S3 Bucket.
- Handler script file
When we bring our container, we should use the SageMaker Inference Toolkit to adapt the container to work with Sagemaker hosting. We have to create the inference code that includes an inference handler, a handler service, and an entry point. [7]
- Main script file
The main.py has the implementation of the entry point. It will start the model server by invoking the handler service after you specify the entry point located in your Dockerfile.
The next step is to create the docker file. We have to copy the main.py, train.py, and the handler.py scripts, install the libraries and specify the Python file main.py as the entry point.
If you are interested in how we create these files, check it in https://gist.github.com/JulianaFontolan/bf71bfb32fc979eeeea758d3eaa8ce5e
Now we can build the docker and push it into the Amazon Elastic Container Registry (ECR):
If you enter the ECR console, you can see see the image tagged as ‘latest’. Now, we can use this container to deploy an endpoint in Sagemaker!
2.2 Model Training
Now that our model container is inside Amazon ECR, we can train it as a prebuilt model from Sagemaker. We have to specify the path on S3 with the training file and another one with the output model result.
We will use the Estimator API [8], which is a high-level interface for Sagemaker training. We should pass some parameters, as you can see bellow:
Now we have a trained model saved at our S3 bucket.
2.3 Model Deploy/Automatization/Code Pipeline
Now that we have our container inference and our trained model, it is time to deploy that solution in an endpoint in Amazon Sagemaker. This example will cover how to consume the model through an API inside Amazon Sagemaker.
For this step, we use the Amazon Codepipeline service. The advantage of using this tool is that you can automatize the deployment process. So, when you have a new model, the pipeline will run automatically.
Automating the model optimization process is still possible, and each new pipeline generates a new model. But I don’t cover that here in the article. Following are the steps that you should proceed:
2.3.1 Choose Pipeline Settings
Here is an effortless step; the main point is to point the Custom Location into the S3 container bucket where you saved your model.
2.3.2 Add Source Stages
The Codepipeline has some kinds of integrations with development tools as Github. The main point here is that they will read the files/code from your repository and start the pipeline with that information.
Codepipeline can use the AWS Cloud Formation advantages, a service that will provide all the necessary configuration to up the resources that you need. We need to describe those resources needed inside the parameters.json and template.yaml files. You can take the standards of these files at this link.
In this example, the source will be Github with those two files given the instructions to the Cloud Formation.
2.3.3 Add Build Stages
This step can be helpful, mainly if you create a code test or have to compile something. I skipped this step because I didn’t need to compile anything or create a test action.
2.3.4 Add deploy Stages
We can choose a lot of services to do a deployment, i.e., CodeDeploy, Amazon ECS. In this example, we will use the AWS Cloud Formation and give all the configuration details that this service needs to know to provide all the resources we need.
2.3.5 Review
The last step before starting our pipeline is to check all that you configured and see if it is necessary to change something. After that, you are ready to create your own pipeline.
3. Have a little fun
It is nice to go to the Cloud Formation console and see all the resources being started by this service.
Also, it is nice to see the endpoint being created inside the Amazon Sagemaker console (this endpoint is where our model was deployed and is available).
And finally, you can call your model using some tool, for example, inside a jupyter notebook:
In this case, the output prediction of the model is two, which means that the phrase “Quero usar seguro” in Portuguese or “I want to use insurance” in English belongs to topic 2 (that we already know is the topic “Activate insurance”).
4. Conclusion
We can see from this test that when we want to bring our code to the platform, it can be challenging to understand the processes of deployment. There are several options and a lot of documentation, and sometimes this ends up not helping to understand the step by step to make the final deployment. Still, the issue of policies and roles is also a challenge since, for each service, there is a type of police needed, which can generate a lot of work to add them. But, when we understand the steps, and the mapped policies, replicating this to other process is easy.
One of the main advantages is that it is simple to set up a machine, and we don’t have to worry about infrastructure issues. We also have connections with Github, host your model as an endpoint to be consumed, and a fast solution to deploy a new model in the service.
Sagemaker is a powerful tool, and I hope this article can help you take advantage of it.I would like to thank my colleagues at Sinch who helped me review the writing.
References
[1] https://aws.amazon.com/pt/solutions/implementations/aws-mlops-framework/
[2] https://docs.aws.amazon.com/solutions/latest/aws-mlops-framework/aws-mlops-framework.pdf)
[3] https://radimrehurek.com/gensim/models/nmf.html
[4]https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.NMF.html
[5] https://aclanthology.org/2021.eacl-demos.31.pdf
[6] https://github.com/MIND-Lab/OCTIS
[7]https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html
[8] https://aws.amazon.com/pt/cloudwatch/
