Unleashing the Power of Azure Open AI and Azure Cognitive Search

Azure Cognitive Search Combined with Azure Open AI

A question that we often hear from our customers is, “How can we build a chatbot that’s powered by our own enterprise data?” For years, Azure Custom Question Answering has been the go-to solution, until — the Generative AI Disruption.

Our curiosity was piqued, and we decided to explore what would happen if we combined Azure Cognitive Search and Azure Open AI Service. The result? An unmatched solution that seamlessly blends the best of both worlds — the superior features of Azure and the remarkable ability of GPT to communicate in natural language.

By integrating these capabilities, enterprises can build powerful applications that effortlessly integrate with their own data, all while providing users with an intuitive and natural interaction experience.

This blog is a deep dive into our experience with this dynamic duo.

Azure Cognitive Search — a cloud-based search-as-a-service platform that provides indexing and querying capabilities for structured and unstructured data. It allows you to add search functionality to your application, making it easier for users to find what they’re looking for.

Azure Open AI — an AI service that provides a set of pre-built models and APIs that you can use to add intelligence to your application. The Open AI models are a collection of generative AI models that can produce language, code, and images.

Benefits of the system

1. Improved Accuracy and Efficiency — Azure Cognitive Search combined with Azure Open AI’s semantic understanding capabilities can help businesses deliver more accurate search results and improve the user experience.
2. Scalability — As your business grows and your chatbot becomes more popular, you can easily scale up your infrastructure to handle increased demand. This ensures that your chatbot remains responsive and effective even as the volume of user inquiries increases.
3. Continuous Improvement — Analytics can be set up to monitor user behaviour and identify areas where your chatbot can be improved.
4. Security — Robust security features, including role-based access control, encryption of data at rest and in transit, and compliance with industry standards and regulations.
5. Ease of use — User-friendly interface and a set of APIs that are easy to use, even for developers who have little experience with search technologies.

Build the system

Architecture

Create the resources required:

1. Log into the Azure portal.
2. Create “Azure Cognitive Search” and “Azure Open AI” from the list of available services.

Create the knowledge base:

To overcome the token limitations for Open AI, it is necessary to break the documents down into manageable chunks (pages and corresponding sections).

Steps to be followed:

1. Use a local pdf parser library or Azure Forms Recognizer to extract text from the documents.
2. Split this text into individual pages, and store each page as a separate item in a list. To keep track of where each page begins in the original document, you should also store the page number and offset.
3. To further facilitate the processing of the documents by Open AI, each page is then uploaded to blob storage. This makes it easier to retrieve the pages as needed for processing.
4. Finally, the pages are indexed in a search index. This allows for easy searching and retrieval of relevant information from the documents.

Code available at: https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/scripts/prepdocs.py

Test the system

Below is a sample code to test the system:

print("\n\nHey there, I am Pixie. How may I help you?")

while True:
    query = input("\n\nUser:")


    search_results = get_search_results(query)
    if len(search_results["value"]) > 0:
        context = ""
        for i in range(min(len(search_results["value"]), top_result)):
            context += search_results["value"][i]["content"]
    else:
        context = "I'm sorry, I don't know the answer to that."

    prompt = f"{context}\n\nAnswer the following question only from the text above and If cannot be answered say I'm sorry, I don't know that.\n\nQ:{query}\nA:"
    response = get_gpt_response(prompt)
    print(f"\n\nPixie: {response}")

Send the user’s query to cognitive search and get the results.

def get_search_results(query):
    url = f"https://{search_service_name}.search.windows.net/indexes/{index_name}/docs/search?api-version={api_version}"
    headers = {
        "Content-Type": "application/json",
        "api-key": {api_key}
    }
    payload = {
        "search": query,
        "queryType": "full",
        "searchFields": "content"
    }

    response = requests.post(url, headers=headers, json=payload)
    return response.json()

Send the search results to Azure Open AI.

def get_gpt_response(prompt):
    response = openai.Completion.create(
        engine = "sample",
        prompt=prompt,
        temperature=0.7,
        max_tokens=256,
        top_p=1,
        frequency_penalty=0,
        presence_penalty=0,
        best_of=1,
        stop=["\n"]
    )
    return response.choices[0]['text'].replace('\n', '').replace(' .', '.').strip()

Output:

Below are some sample screenshots showcasing the power of the system.

Conclusion

The combination of Azure Cognitive Search and Azure Open AI Service provides an unmatched solution for enterprises looking to build powerful chatbot applications that can communicate with users in natural language and provide intuitive and personalized interactions. This is very relevant in today’s digital world, where customers demand quick and efficient responses to their inquiries. With the ability to continuously improve through analytics and scale to handle increased demand, chatbots built with this integration can revolutionize the chatbot landscape and help businesses achieve their goals in a fast-paced digital world.

About the Author

Geethu Suresh is a Microsoft .Net Consultant at Version 1.

References:
https://github.com/Azure-Samples/azure-search-openai-demo