Deploying Your Own Embedder Model on watsonx Platform Using IBM Watson Machine Learning — featuring Thai Language
In the world of Retrieval-Augmented Generation (RAG) applications, embedder models are crucial. These models convert text into dense vectors, enabling effective retrieval and comparison of information. While numerous embedder models are available, not all fit every use case. For instance, many cloud services offer a limited selection of embedder models, which might not be localized or generalize well to all languages. Hence, the ability to deploy your own embedder model becomes essential.
In this article, we will explore how to deploy a custom embedder model, specifically the simcse-model-phayathaibert
created by kornwtp, on the Watsonx platform using Watson Machine Learning (WML) deployments.
Steps to Deploy Your Embedder Model
Let’s dive into the steps involved in deploying the simcse-model-phayathaibert
on the Watsonx platform using Watson Machine Learning.
Prerequisites
- IBM Cloud account
- Watson Machine Learning service instance
- Hugging Face model (
simcse-model-phayathaibert
)
Step 1: Install Required Libraries
First, ensure you have the necessary libraries installed in your environment:
!pip install sentence-transformers==3.0.0
!pip install -U ibm-watson-machine-learning
Step 2: Download and Prepare the Model
Define the function to use the embedder model
import numpy as np
import requests
from ibm_watson_machine_learning import APIClient
def my_embedding_function():
from sentence_transformers import SentenceTransformer
model_name = 'kornwtp/simcse-model-phayathaibert' # Replace with the desired model name
try:
model = SentenceTransformer(model_name)
except Exception as e:
return {"error": str(e)}
def score(payload):
# we assume only one batch is sent
sentences = payload['input_data'][0]['values'][0]
try:
embeddings = model.encode(sentences)
return {
'predictions': [
{
'fields': ['sentence', 'embedding'],
'values': [[sentence, embedding.tolist()] for sentence, embedding in zip(sentences, embeddings)]
}
]
}
except Exception as e:
return {"error": str(e)}
return score
# Example usage:
embedding_function = my_embedding_function()
# Example payload
payload = {
'input_data': [
{
'values': [
["กลุ่มผู้ชายเล่นฟุตบอลบนชายหาด", "กลุ่มเด็กชายกำลังเล่นฟุตบอลบนชายหาด"]
]
}
]
}
result = embedding_function(payload)
print(result)
Step 3: Configure Watson Machine Learning
Authenticate and set up your Watson Machine Learning (WML) service:
url='https://us-south.ml.cloud.ibm.com'
api_key='your-api-key'
wml_credentials = {
"url": url,
"apikey": api_key
}
client = APIClient(wml_credentials)
space_id = 'your_space_id'
client.spaces.list(limit=10)
client.set.default_space(space_id)
Creating a deployment space
%%writefile environment.yml
channels:
- empty
- nodefaults
dependencies:
- pip:
- sentence-transformers==3.0.0
meta_props = {
client.package_extensions.ConfigurationMetaNames.NAME: "transformers",
client.package_extensions.ConfigurationMetaNames.TYPE: "conda_yml"
}
pkg_extn_details = client.package_extensions.store(meta_props, "./environment.yml")
pkg_extn_id = client.package_extensions.get_id(pkg_extn_details)
Step 4: Deploy the Model
Create a software specification:
base_id = client.software_specifications.get_id_by_name("runtime-23.1-py3.10")
meta_props = {
client.software_specifications.ConfigurationMetaNames.NAME: "default with sentence-transformer",
client.software_specifications.ConfigurationMetaNames.PACKAGE_EXTENSIONS: [{'guid': pkg_extn_id}],
client.software_specifications.ConfigurationMetaNames.BASE_SOFTWARE_SPECIFICATION: {'guid': base_id}
}
sw_spec_details = client.software_specifications.store(meta_props)
sw_spec_id = client.software_specifications.get_id(sw_spec_details)
Store function:
function_props = {
client.repository.FunctionMetaNames.NAME: 'simcse embedder-v2',
client.repository.FunctionMetaNames.SOFTWARE_SPEC_UID: sw_spec_id
}
function_details = client.repository.store_function(my_embedding_function, function_props)
function_id = client.repository.get_function_id(function_details)
print(function_id)
Create a deployment for the model, once deployed it will show up in your space:
hardware_spec_id = client.hardware_specifications.get_id_by_name('M')
deployment_props = {
client.deployments.ConfigurationMetaNames.NAME: 'simcse embedder deployment v3',
client.deployments.ConfigurationMetaNames.ONLINE: {},
client.deployments.ConfigurationMetaNames.HARDWARE_SPEC: { "id": hardware_spec_id, 'num_nodes': 1}
}
deployment_details = client.deployments.create(function_id, deployment_props)
deployment_id = client.deployments.get_id(deployment_details)
payload = {
'input_data': [
{
'values': [
["กลุ่มผู้ชายเล่นฟุตบอลบนชายหาด", "กลุ่มเด็กชายกำลังเล่นฟุตบอลบนชายหาด"]
]
}
]
}
result = client.deployments.score(deployment_id, payload)
print(result)
{'predictions': [{'fields': ['sentence', 'embedding'],
'values': [['กลุ่มผู้ชายเล่นฟุตบอลบนชายหาด',
[1.1157499551773071,
-0.1111806258559227,
-0.5575138330459595,
0.4732488691806793,
0.4931944012641907,
-0.0910203829407692,
0.861914873123169,
1.1215389966964722,
1.0791149139404297,
-0.08809694647789001,
0.613644540309906,
0.9309089183807373,
0.372568279504776,
-0.3043205738067627,
-0.48598262667655945,
0.3027377128601074,
1.9552265405654907,
-0.04576759785413742,
-0.5975145697593689,
0.34542346000671387,
-0.3229110836982727,
0.07778410613536835,
1.0909312963485718,
...
0.4387320876121521,
-0.39540353417396545,
0.1730499118566513,
-0.8454257845878601,
0.9247894883155823]]]}]}
This response will include the embedded vector for the input text, demonstrating that your custom embedder model is now successfully deployed and operational on the watsonx platform.
Extra
You can also https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-deploy-functions_local.html?context=cpdaas&locale=it#store-wmls read more here to see parameters such as `client.hardware_specifications.get_id_by_name`. Here I used size M which is 4vCPU and 16 GB RAM. Also can adjust the num_nodes for more instances.
Conclusion
Deploying a custom embedder model on the Watsonx platform using Watson Machine Learning allows you to leverage localized and specialized models for your RAG applications. By following these steps, you can ensure that your applications have access to high-quality, tailored embeddings, improving their overall performance and reliability.