Part 1.1 : Drag an drop ML, say hi to AWS Sagemaker
Hi, I am Mine, recently I was working on a project about forecasting and data analysis. Data was coming from one of our internal project, we using it as a project management tool. There is lots of modules on the project of course, but i was working on data of timesheets. Its a daily record of which project that i worked on daily basis and home-office requets. We were trying to create PoC to forecasting the projects feature, what type of projects are more successful, etc.
I started to look for a service on AWS obviously because as a backend developer, as you can imagine, the knowledge that i know was very little even if I personally curious about ML and read about the topic. Hands on practice was little bit confusing that I expected. Solution architecture and where I am hading changed a lot (i mean, a lot) during the development process. A fresh piece of knowledge that I read was completely changing my perspective about what I can do and what I can’t. Sagemaker as a service, langchain as a library, huggingface models, how to run everyting smootly on aws from code to a endpoint.. there was lots of things to learn.
Project was a building a question-answering model for giving responses to the question over the data that we have.
I can’t say that I know everything now on, actually really far from that. Yet I have pretty good level of experimentation and still I think it will be very helpful for someone like me. Because even if that wasn’t a brightest solution at that moment, I will mention about everything that i think , I learned and did at that moment to solve the problem. Because even there are lots of knowledge out there when you start to read and search about spesific topics, you find yourself in a rabbit hole of github issue discustions.
Really, I didn’t want to make a big entrence but there are lots of think that I want to cover, I will divide this in to a small chunks, actually every part will end on a milestone desicion and we will be move from there to the end result.
Shall we...
Attention pls, don’t be bold , check your cost !
Okey, before I start to talk technical I want to mention something. You can use lots off pre-trained model that you can find in HuggingFace on AWS Sagemaker within a notebook-instance.
Know your needs and don’t try to use fanciest models like Falcon. Because of ‘I want to try’, I had to use a really big notebook instance and as you can imagine cost is getting its x’s so quickly. Just be aware of that, I wasn’t, after 2 day pasts I realized that Sagemaker was on the top of the cost list :)
What is AWS Sagemaker?
Amazon SageMaker provides machine learning (ML) capabilities to prepare, build, train, and deploy ML models. There is lots of tool in it for doing that. You can start with very little knowledge of ML on AWS Sagemaker.
You can create your own notebook-instance and deploy an endpoint and reach it via Lambda or API Gateway.
You can use its Sagemaker Studio for more structural flow, its basicly an IDE for ML. You can fulfill probably all your needs inside it.
And there is Canvas, after trying to run my own notebook instance as i mentioned earlier on cost section above, one of my colleague came with its name, I realized that Canvas was all I need from start. Because I had almost no experience, beyond technical read, with ML, algorithms, LLMs etc.
Canvas is a tool that you can make predictions with no-code requirement. It is like a tool that you use for teach code to kids actually. You don’t have to know any algorithm or ML knowledge to start, but if you want to go further, of course, you should do some research and how to prepare your data and what is the differences between algorithms and which one to apply.
How to setup your enviroment?
You need to have an AWS account of course, from console go to Sagemaker’s dashboard. And you can see Canvas on the left menu.
For to start, you should create a Sagemaker Domain, I select the quick setup, didn’t change anything.
And after the creation you can open Canvas with a IAM role that you newly created.
Let’s s check out whats in it.
There is sections that we will use a long the way, but I want to start from the top and mention each of them.
Ready-to-use models: You can start generating predictions with pre-built models as you can imagine. To get started, bring your data such as text, images, or documents and select a model to meet with your case.
There are lots of models for basic use-cases, such as : sentiment analysis, language detection, object or text detection in images and document queries. Actually document queries is such a powerful model to creating a simple question-answering model.
Before to move My Models section, we need to import our dataset via Datasets section so I am jumping there.
Datasets: You can see there is few sample dataset on .csv format. You can bring your own with using create button.
There 3 types of datasets, if you want to work on images or document go with that section but our data is a tabular (aka structural text). After you gave a name to your dataset, you can upload data from various of sources.
You can go with local upload too but I will go with S3 and select my bucket and import from S3 to Canvas.
From now on we will create our model, try to clean-up data. If you click create a model button this will be on your page:
As I said my datasets doesn’t include any images and Canvas will not let me to select it also. I can go with predictive or text analysis. What I want to do is predict the feature so I will go with predictive analysis but if your context includes more classification you can continuw with text analysis.
Now we import our data and created a model, lets move to the My Models section.
Model type’s >> Forecasting
This the building tab, we can change column to predict and select a model type for your analysis. I select the project_id and for somehow canvas didn’t let me the choose forcasting model-type. It was recommending 3+ category model type. I was like k lets try it but was curios about why my data didn’t fit for forecasting.
After that, i click to quick-build and waited to finish. It took like 3–4 min, but it can took longer if your data is to large. When it done with building, analyze page welcoming us. You can check your accurency and do some data engineering if you need it. Lets predict !
There is 2 types of prediction, batch and single, i go with single and select the parametres. Waited a little bit and here my prediction.
Okey its seems nice I can predict the future but my mind was on why I couldn’t select the forecasting in the beggining. Because categorical prediction can be performed on data that can be put into a category, but i want to actually forecast the future.First I think that the probem was on my data’s structure.
First of all, you can do changes on your data over Canvas too, if you want to find out what you can do click here.
Then, I started to search for ‘how to prepare your data for forecasting?’ an here is what I found : We should have time-series data for historical events. And there are few features (columns) to be key information for forcasting.
- item_id, which is a unique identifier for the things you want to forecast. The unique identifier can represent distinct products, services, customers and more. This feature communicates “what”.
- timestamp, when the historical event occurred. This feature communicates “when”.
- target_value, expresses the quantity of the event. These continuous values can be integer or decimal (float) numbers. This feature communicates “how much”.
(I will put the links to refer on the end)
As you can see from the picture above, I have email address (item_id), date (timestamp), hours (target_value). But somehow it wasn’t let me to choose forecast. I tried on AWS Forecast service as well, didn’t worked very well either.
Later on, I learned that you have to give permission for your IAM Role when you create a domain. You can find it in the Admin Configurations > Domains > your_domain > user_profile > Canvas Settings , if you click the edit button you will see time-series forecasting configuration. Basicly you use AWS Forecast on backend and you have to access the service via the role you have for Sagemaker. For more information go here.
When we finish this , there was a question poped up on my mind, how I can chat with this model. I had already have a model for forcasting, now I will need a chat-base model to talk and somehow connect this two of them.
And I came accross with multi-model deployment at that point, I will give a really little information about it because I actually didn’t implement it, yes we came to a milestone desicion, but before that let me give you little information about multi-model deployment. Even before that, I should mention how we deploy a single model on Sagemaker.
Model Deployment
I will seperate this ‘model’ into 2 categories, first one is the model that you create through Canvas, second example will be deploying a Huggingface model. Because I had the use chat-base model to actually talk with it.
1-Deploy model from Canvas
You can access the model deployment option through any of the following methods:
- On the My models page of the Canvas application, you can choose the model that you want to deploy. Then, from the model’s Versions page, you can choose the More options icon next to a model version and select Deploy.
- When on the details page for a model version, on the Analyze tab, you can choose the Deploy option.
- When on the details page for a model version, on the Predict tab, you can choose the More options icon at the top of the page and select Deploy.
- On the Operations page of the Canvas application, you can choose the Deployments tab and then choose Create deployment.
Within that our model will be deployed and we can see it on Sagemaker Studio.
Launch Sagemaker Studio and go to Model Registery. You will see your models, I deployed 2 of them.
2-Deploy model from Huggingface
Go to huggingface site and open you models details, for me I was thinking that LaMini will be a good fit for my problem. Click here to go its page.
On deploy button you will see options to where to deploy model. Select Sagemaker endpoint and it will give you a code snippet.
import json
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uri
try:
role = sagemaker.get_execution_role()
except ValueError:
iam = boto3.client('iam')
role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']
# Hub Model configuration. https://huggingface.co/models
hub = {
'HF_MODEL_ID':'MBZUAI/LaMini-Flan-T5-783M',
'SM_NUM_GPUS': json.dumps(1)
}
# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
image_uri=get_huggingface_llm_image_uri("huggingface",version="1.1.0"),
env=hub,
role=role,
)
# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.g5.2xlarge",
container_startup_health_check_timeout=300,
)
# send request
predictor.predict({
"inputs": "how can I become more healthy?",
})When we call deploy(), it will create Sagemaker endpoint for you, you can see it on AWS Sagemaker Dashboard > Inference > Endpoints.
Multi-Model Deployment
Multi-model endpoints are ideal for hosting a large number of models that use the same ML framework on a shared serving container.
There a 2 approach to achive multi-model deployment. I will not deep dive on this topic because, as I told you, I didn’t have hands-on practice yet I will be put some useful links below.
Just to know that it exist, if its fit your case you can go with links that i will put below, I believe they will be helpful.
1- Host multiple models in one container behind one endpoint
(Title says everything)
Go here to find related docs.
2. Host multiple models which use different containers behind one endpoint
(Title says everything again)
Go here to find the related docs.
We came to end of part 1. For the readers that want to continue, milestone desicion was, not to create seperate models for chat and the data. I found this solution was too complicated and hosting two different models on AWS will not be so budget friendly. We will use Langchain to connect our data to a chat-base model.
Stay tuned !
