In this article, we will detail how to deploy davinci003 model in Azure and integrate it in Microsoft Teams through an outgoing webhook. Embedding OpenAI APIs, such as the Davinci003 model, into Microsoft Teams can bring a wide range of benefits to teams and organizations. By integrating advanced natural language processing and text generation capabilities, teams can automate tedious tasks such as document creation and language translation, allowing them to focus on more important tasks. Additionally, the integration can also enhance the overall communication and collaboration experience by providing more accurate and efficient responses to questions and inquiries. Furthermore, by providing the ability to generate automated responses to common customer support queries, teams can improve their customer service and increase customer satisfaction. Additionally, it can help to automate repetitive tasks and improve the overall efficiency and productivity of a team. Overall, the integration of OpenAI APIs into Microsoft Teams can bring significant value to any organization by streamlining workflows, improving communication and collaboration, and increasing overall efficiency.
This is the first article in a series where we will integrate the OpenAI APIs with different tools and for different purposes to improve the efficiency of your processes.
Introduction to Azure OpenAI Service
Azure OpenAI Service is a cloud-based platform provided by Microsoft Azure that allows developers and organizations to easily build, deploy, and manage applications that utilize the advanced artificial intelligence (AI) capabilities of OpenAI. The service provides a set of pre-built AI models and a set of tools to help developers quickly and easily incorporate these models into their applications. It also allows developers to tune their own models using OpenAI’s powerful GPT-3 technology, and embed them into their applications through REST API calls. This service can be particularly useful for developers and organizations that want to take advantage of the latest advancements in AI technology without having to invest in expensive hardware and infrastructure.
As you might know by now, Microsoft released earlier this month the General availability of Azure OpenAI Service, and some days after they announced the third phase of our long-term partnership with OpenAI through a multiyear, multibillion-dollar investment to accelerate AI breakthroughs to ensure these benefits are broadly shared with the world.
The Azure OpenAI service provides REST API access to OpenAI’s powerful language models including the GPT-3, Codex, and Embeddings model series. To access the service you can follow the documentation here.
GPT-3 models are a series of models that can understand and generate natural language. As we can see in the diagram the different models have different capabilities going from more simple to more complex. This does not mean you will always have to choose the most complex models. Simpler models are optimized and will give better results for certain use cases. Additionally, these will have lower inference times.
- Ada: Simple classification, parsing, and formatting of text
- Babbage: Semantic search ranking and moderate complex classification
- Curie: Answering questions and complex nuanced classifications
- Davinci: Summarizing for specific audiences and generating creative content.
Codex models are a series of models that can understand and generate code, including translating natural language to code. While Davinci is the most capable Codex model, Cushman will be a very useful model for many code generation tasks, it will additionally run faster and will be cheaper than Davinci.
Lastly, we have the Embeddings models that can understand and use embeddings. For those of you reading and not knowing what is an embedding, it is a way of representing a discrete variable, such as a word, a sentence, or an image, as a continuous and dense vector of real numbers. The goal of this representation is to capture the underlying structure or meaning of the variable in a way that is useful for various machine learning tasks. So words with similar meanings will have similar embeddings.
Currently, there are three families of Embeddings models for different functionalities: similarity, text search, and code search. You could also imagine using the Embedding models together with the completion models in order to do more complex question answering scenarios. An example can be found in the repo here.
Step-by-step tutorial
1. Deployment of the davinci003 model
We will start by going to the Microsoft Portal and creating an Azure OpenAI service with the name you want. Please contact your Azure Administrator or refer to the documentation on how to ask for access to Azure OpenAI service if you do not see it in your available services.
Once the service is up and running you will go to the service and deploy your own model. In our case, we will be deploying a text-davinci003 model.
Once deployed you will be able to see it in the Model Deployments tab.
Later on, you will have to come back to fetch the keys and endpoint that will be embedded in your code.
2. Create a Python API to communicate with Microsoft teams
For the purpose of this demo, we will create a small flask application with a single endpoint that will be triggered by the outgoing webhook in teams.
Following here you will find an example code for such an application:
from flask import Flask, render_template, request, redirect, url_for, send_from_directory
from flask import request, jsonify
from bs4 import BeautifulSoup
import os
import requests
import json
import openai
app = Flask(__name__)
openai.api_key = "<FILL IN WITH YOUR OWN API KEY>"
openai.api_base = "<FILL IN WITH YOUR OWN ENDPOINT>"
openai.api_type = 'azure'
openai.api_version = '2022-12-01'
deployment_id='<FILL IN WITH YOUR OWN deployment id>'
def openai_chat_call(text):
response = openai.Completion.create(
engine=deployment_id,
prompt=text,
temperature=0.9,
max_tokens=150,
top_p=1,
frequency_penalty=0.0,
presence_penalty=0.6,
stop=[" Human:", " AI:"]
)
return response
def extract_text(attachments):
text=""
for item in attachments:
if item["contentType"]=="text/html":
text = text + item["content"]+" "
return text
def clean_html(html):
soup = BeautifulSoup(html, 'html.parser')
text_nodes = soup.findAll(text=True)
return ' '.join(text_nodes).replace("teamsbot", "", 1)
@app.route('/gpt3', methods=['POST'])
def teams_webhook():
content = request.json
html_message = extract_text(content["attachments"])
message = clean_html(html_message)
try:
res = openai_chat_call(message)
res_text = res['choices'][0]['text'].replace('\n', '').replace(' .', '.').strip()
message = jsonify({'type': 'message','text':res_text})
except Exception as e:
message = jsonify({'type': 'message','text':'There was an error:'+str(e)})
return message
if __name__ == '__main__':
app.run()Note that you can tweek the OpenAI parameters in order to better fit your specific use case.
3. Creation of an outgoing webhook in teams
An outgoing webhook in Microsoft Teams is a feature that allows a team to create a custom bot that can send messages to a specified URL, such as a web application or a service running in the cloud. The webhook is triggered by a specific command or phrase that is entered into a Teams channel (in our case @teamsbot), and it can be used to perform a wide range of tasks, such as sending notifications, triggering actions, or providing information. The web service can then respond with rich messages, which can include cards and images. Sending richer messages containing images could be useful in the case that you would like to connect other APIs such as the DallE API.
To create an outgoing webhook you will need to go to the “Manage team” menu. Once in there, you will have to navigate to apps.
An option will then appear on the button of the menu to create an outgoing webhook. Lastly, you will have to give it a name (this name is the one that will be used to call your webhook, i.e. in our case we will call it teamsbot). The callback URL will be the URL of your flask service together with the endpoint that we developed in flask (in our case the endpoint would be https://your-service.com/gpt3 ). Note that the URL will need to be exposed through HTTPS, so if you are running your application locally, one thing you could potentially do to expose your application through HTTPS would be to expose it through ngrok (ngrok is a tool that allows you to securely expose a local web server to the internet) and use the URL it will provide you.
Additionally, while creating your outgoing webhook you can also give it a description and a profile picture. Once you have completed all the necessary information you can create your webhook. For more information on how to create outgoing webhooks, you can go here.
You can now ask questions to your davinci003 model by tagging your outgoing webhook in your Teams conversation (@teamsbot)
Bonus
If you would like to deploy your flask application in the cloud you could easily do this by creating an Azure App Service. In the following URL you will find a tutorial on how to do this:
Additionally, if you deploy your application in Azure you could store your API key in Azure Keysvault and use a managed identity on your Azure app services to access it.
Another additional way of improving the security of your application would be by using the Hash-based Message Authentication Code (HMAC) that will appear when creating your outgoing Webhook in Teams. This is a security token used to authenticate calls between Teams and the designated outside service. The HMAC security token doesn’t expire and is unique for each configuration. You could build a method to verify the Outgoing Webhook HMAC token to ensure that any API calls arriving at your service would be coming from your Teams outgoing webhook. This method will have to use the value “HMAC 03TCao0i55H1eVKUusZOTZRjtvYTs+mO41mPL+R1e1U=” in the authorization of the request header. For more information about how to use HMAC security tokens, please refer to the documentation.
I hope this tutorial will be useful. Please do not hesitate to contact me in case you would have any questions, suggestions for improvements, or future articles for the series.
