Getting started with Watsonx.ai V
This article is part of the series created and curated by me, Getting started with Watsonx.ai: https://medium.com/@nathalia.trazzi/list/watsonxai-series-english-08947bbd62f2
To continue, it is important that you have programming knowledge, understand the basics of Watsonx.ai.
In the previous article (https://medium.com/@nathalia.trazzi/getting-started-with-watsonx-ai-iv-015f14db5236), it was shown how to access Watsonx.ai through a notebook generated by the Prompt Lab. In this article, it will be demonstrated how to work with Watsonx.ai via API in a different way.
Getting started with Watsonx.ai API
- Open your preferred software; The one used here will be Visual Studio Code.
- Next, create a file with the extension
.pyor.ipynb. Import the necessary dependencies to communicate with Watsonx.ai.
from ibm_cloud_sdk_core import IAMTokenManager
import requestsUnlike the previous article, the connection to Watsonx.ai will be established using Python’s requests library. The ibm_cloud_sdk_core library allows the connection through the token generated by the IBM Cloud API key.
The function authenticator_ibmcloud passes the apikey to IBM Cloud and the url to establish this connection, and then returns the token by calling the get_token() function.
def authentificator_ibmcloud(apikey, url):
"""
Function responsible for authenticating with IBM Cloud.
args:
apikey: IBM Cloud API key
url: URL to connect to IBM Cloud.
returns:
token to establish the connection
"""
token_manager = IAMTokenManager(apikey=apikey, url=url)
return token_manager.get_token()The create_headers function will be responsible for creating a header to communicate with Watsonx.ai through the token created with the authentificator_ibmcloudfunction.
def create_headers(acess_token):
"""
Function responsible for creating a header to establish the connection:
args:
access_token: token from the authenticator_ibm_cloud function
returns:
Authentication
"""
return {
"Authorization": "Bearer " + acess_token,
"Content-Type": "application/json",
"Accept": "application/json"
}The next code block will consist of apikey_cloud, url_cloud, and url_wx variables.
apikey_cloud: Paste your IBM Cloud API key in this variable.
If you don't know how, access Part VI of this article at the link, where there is a step-by-step guide at the end of the article on how to do this: https://medium.com/@nathalia.trazzi/getting-started-with-watsonx-ai-iv-015f14db5236
The values of the variables url_cloud and url_wx are fixed, keep them as they are.
apikey_cloud = ""
url_cloud = "https://iam.cloud.ibm.com/identity/token"
url_wx = "https://us-south.ml.cloud.ibm.com/ml/v1/text/generation?version=2023-05-29"Now let’s call the function authenticator_ibmcloud to generate the token and then create_headers to create the communication header.
acess_token = authentificator_ibmcloud(apikey_cloud, url_cloud)
headers = create_headers (acess_token)In the next step…
project_id_wx: Fill in project_id_wx with your project number.
If you don’t know how, access Part VI of this article at the link, where there is also a step-by-step guide at the end of the article on how to do this: https://medium.com/@nathalia.trazzi/getting-started-with-watsonx-ai-iv-015f14db5236
Variable model: This is the Foundation model chosen to generate a response. You can keep the same value or replace it with the chosen model.
All available Foundation models on Watsonx.ai: https://www.ibm.com/products/watsonx-ai/foundation-models, https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx
Variable question: This is the question that will be sent by the model. You can also send another one of your preference.
project_id_wx = ""
model = "meta-llama/llama-2-70b-chat"
question = "Who is Taylor Swift?"The function below will generate the response. The following values are passed as parameters to this function: question (the question to be sent), the project_id of the project, and the model_id.
Note that the construction of the prompt begins after the "input" field, where the model is asked to respond to the question in a humanized and empathetic manner.
Instructions for the model to respond only about fruits and vegetables and not to respond about public figures and celebrities were included. Then, there are two examples of how the model should respond to the questions.
def generate_data(question, project_id, model_id):
"""
Function responsible for getting the prompt, receiving the question.
args:
question: User's question.
project_id: ID of the Watsonx.ai project.
model_id: Chosen Foundation Model.
"""
return {
"model_id": model_id,
"input": f"""
<<SYS>>
[INST]
[Instructions]
Answer to {question} in a humanized and empathetic manner. You only answer questions about fruits and vegetables; do not answer anything about public figures or celebrities.
[/Instructions]
[Examples]
Question: What is a tomato?
Answer: The tomato is the fruit of the tomato plant (Solanum lycopersicum; Solanaceae). Its family also includes eggplants, peppers, and bell peppers, which are part of the Solanaceae family, along with some non-edible species. It first appeared in the press in 1595.
Question: Who is Britney Spears?
Answer: I do not answer questions about public figures.
[/Examples]
[/INST]
Question: {question} <<SYS>>
Answer:
""",
"parameters": {
"decoding_method": "greedy",
"max_new_tokens": 60,
"min_new_tokens": 20,
"stop_sequences": ["."],
"repetition_penalty": 1
},
"project_id": project_id
}As parameters for the model, I am passing “greedy”, with a maximum number of tokens of 60 and a minimum of 20. If after spending 20 tokens it encounters a “.”, the model will immediately stop generating content.
The last function, send_request, receives the parameters to send a request to Watsonx.ai and returns the response.
def send_request(url, data, headers):
"""
Function responsible for sending the request to Watsonx.ai.
args:
url: URL for connecting to Watsonx.ai
data: result from the generate_data function of Watsonx.ai
headers: result from the Watsonx.ai authentication call
returns: response from Watsonx.ai
"""
response = requests.post(url, json=data, headers=headers)
return response.json()['results'][0]['generated_text']
data = generate_data(question, project_id_wx, model)
response = send_request(url_wx, data, headers)Then print the response to see the result:
print (response)The result should be something like the example below:
Changing the question:
question = "What are onions?"And the result will be something like this following:
Note that in the send_request function, there is the following line: return response.json()[‘results’][0][‘generated_text’]
response.json(): This converts the response received from the request into a JSON format so that it can be easily manipulated by the code.['results']: This accesses the 'results' key in the returned JSON, which contains the results of the request.[0]: This accesses the first element of the list of results. Since we typically expect only one response from the model, we assume that the desired response is in the first position.['generated_text']: This accesses the text generated by the model, which is the response to the question sent.
Response generated in the model without filtering the results:
{'model_id': 'meta-llama/llama-2-70b-chat', 'created_at':
'2024-05-23T20:55:52.726Z', 'results': [{'generated_text':
" Oh, onions! They're such fascinating and versatile vegetables,
aren't they? Belonging to the Allium genus and Allium cepa species,
onions are essentially bulbs that grow underground.",
'generated_token_count': 52, 'input_token_count': 221,
'stop_reason': 'stop_sequence'}], 'system': {'warnings':
[{'message': 'This model is a Non-IBM Product governed by a
third-party license that may impose use restrictions and other
obligations. By using this model you agree to its terms as identified
in the following URL.', 'id': 'disclaimer_warning', 'more_info':
'https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx'}]}}That’s all for today… See you in Part VI (When it’s available, though)
