Exploring Azure Open AI for deploying an intelligent “Blob-Chat” Travel Application
Introduction: In an era defined by the rapid evolution of technology, artificial intelligence has emerged as a transformative force. Azure Open AI Studio, a robust platform within the Azure ecosystem, stands at the forefront of this revolution. This article delves into the myriad capabilities of Azure AI Studio, with a specific focus on Blob Chat, demonstrating its prowess through the intelligent analysis of a dynamic dummy-document that encapsulates a travel itinerary. The integration of the “Azure Open AI” capabilities adds an extra layer of intelligence to our conversational application thus making analysis easier in a Chat way (Natural Language Processing) rather than complex data-analytics queries.
In this article we will perform a demo for Blob-Chat Travel Application, with a focus on its configuration, integration and overall deployment from the Azure Portal
Azure AI Studio Overview:
Azure AI Studio is more than just a development platform; it’s an ecosystem that empowers both seasoned developers and budding data scientists.
Let’s embark on a detailed exploration of its key components:
Playgrounds:
- Chat Playground: Beyond a mere testing ground for chat-based models, the Chat Playground is a dynamic environment where conversational AI is molded. Its adaptability allows developers to craft and refine language models that can comprehend and respond to user queries effectively.
- Completion Playground: Tasked with the responsibility of text completion, this playground excels in scenarios where the generation of coherent and contextually relevant text is paramount.
- DALL·E Playground: A canvas for creativity, the DALL·E Playground generates images from textual descriptions. This capability extends the boundaries of AI applications into the realm of visual storytelling.
- Whisper Speech-to-Text: Transcribe your audio using the Whisper model, Use text prompts to improve transcription quality.
- Bring your own data: Connect and ground your data. Deploy to a Web-App or a Copilot via Microsoft Copilot Studio
Management:
- Deployments: The journey doesn’t end with model creation; it extends to deployment management. Explore the intricacies of deploying models, ensuring seamless integration into diverse applications.
- Models: Each model within Azure AI Studio possesses a distinct identity, with properties defining its behavior. Dive into the specifics of model management, understanding properties such as model name, version, update policies, deployment types, and rate limits.
- Data Files: The lifeblood of AI models lies in the datasets they are trained on. Discover how Azure AI Studio facilitates the uploading and management of datasets, providing a foundation for robust model training and testing.
- Quotas: View your quota by subscription and region and track usage across your deployments. Quota is required to create deployments and allows you to flexibly size them according to your traffic needs.
- Content filtering configuration: Azure OpenAI Service includes a content management system that works alongside core models to filter content. Content filtering configurations can be created within a Resource and assigned to Deployments.
Azure Open AI Search Service:
In the vast landscape of data management, Azure Open AI Search Service emerges as a beacon of efficiency. Build a full-text search experience with AI and semantic ranker. Get started building a full-text search experience and learn how to integrate with your custom applications and other Azure services.
Knowledge Center: To quickly educate ourselves about the whole ecosystem of Azure Open AI Search Service.
- Connect your data: Azure AI Search allows you to quickly and easily connect to any data source using either the Import Data workflow or the REST API.
- Adding and Modifying AI Skills: You can add AI skills to your indexers to extract text from images, blobs, and other unstructured data sources. You can use built-in skills from Microsoft or embed external processing into a custom skill that you create.
- Explore Your Data: You can integrate powerful search experiences for your apps using the developer APIs and tools we’ve provided. Use capabilities such as faceting and filtering to help guide navigation. Boost search results using scoring profiles and order-by. Create synonym-maps and more to build a customized and unique search experience.
- Optimizing Performance and Service Administration: You can easily scale Azure AI Search to adjust to the needs of your workload, such as at specific times of the day. Beyond learning when and how to scale, you’ll learn about monitoring, securing your index, and using diagnostic tools.
1. Monitoring Tab: The main Overview page shows query volume, latency, and whether the service is under pressure.
2. Activity log: Reports on actions undertaken by Resource Manager: service availability and status, changes to capacity, and API key-related activities.
3. Monitoring Settings: Provides configurable metrics visualization, alerts and diagnostic logs.
Lets Delve into its components:
Search Management:
- Indexes: The heart of efficient data retrieval, indexes dictate how data is organized for optimal searching. Uncover the nuances of creating indexes that facilitate swift and accurate queries.
- Data Sources: The origin of data plays a pivotal role. Understand how data sources are defined, specifying where the data is located and ensuring a seamless integration into the search functionality.
- Aliases: An index alias is a secondary name that can be used in place of an index’s actual name. You can use the alias for querying, indexing, and other operations, rather than the index name.
- Skillsets: Elevate your search capabilities with skillsets. These predefined skills enhance the understanding and processing of search queries, unlocking a new dimension of efficiency.
- Debug Sessions: Debug sessions in the Azure Portal provide a holistic visualization of any skillset. Using this tool you can drill down to specific steps and easily see where an action might be failing.
Settings:
Fine-tune your search functionalities with a deep dive into settings and classical azure managed-services configurations for Keys, Identity, Networking, Properties etc. These are not mere features but strategic tools in shaping the intelligence of your applications.
- Semantic Ranker: Semantic ranker uses deep neural networks to provide relevant results and answers based on semantics, not just lexical analysis. Surface the most relevant results based on search intent rather than just keywords.
- Search Traffic Analytics: Visualize search metrics and trends in Power BI reports. Reports show top searches, zero-click searches, and trends useful for improving the search experience. Application Insights is required for collecting telemetry from your application. After configuration, you can generate reports on demand.
- Knowledge Center: To quickly educate about the whole ecosystem of Azure Open AI Search Service. (as explained in the above section: Azure Open AI Search Service:)
Preparation Workflow: Hands-On Implementation via Azure Portal
Before directly about the demo, lets prepare the stage for this workflow. And to do that lets get to know about the services that we will use to integrate and configure for deploying an intelligent “Blob-Chat” Travel Application.
Chat Playground:
The Chat Playground, being the epicenter of our Blob Chat application, warrants a comprehensive exploration (already explained in the above section “Azure AI Studio Overview”/Playgrounds)
Advanced Settings:
- Limiting Response to Data Content: The content of responses can be finely tuned. Examine the intricacies of this setting, understanding how it allows you to specify the type and amount of data content in responses, ensuring a more controlled and user-centric experience.
- Strictness: The ability to control the model’s adherence to instructions is a powerful feature. Uncover the impact of strictness on the model’s behavior and learn to leverage it for tailored conversational experiences.
- Retrieved Documents: Limiting the number of documents retrieved during a conversation is crucial for efficiency. Delve into the nuances of this setting, understanding how it shapes the responsiveness and relevance of the AI model.
Configuration:
Deployment
- Deployment: Select the Deployment Type that you created. Currently, Deployment supports 3 models, ‘gpt-35-turbo’. ‘text-embedding-ada-002’ and ‘whisper’.
- Past messages included: Select the number of past messages to include in each new API request. This helps give the model context for new user queries.
- Current token count: This is an estimate of the number of tokens that will be used for the next request.
Parameters
- Max response: Set a limit on the number of tokens per model response. The API supports a maximum of MaxTokensPlaceholderDoNotTranslate tokens shared between the prompt (including system message, examples, message history, and user query) and the model’s response. One token is roughly 4 characters for typical English text.
- Temperature: Controls randomness. Lowering the temperature means that the model will produce more repetitive and deterministic responses. Increasing the temperature will result in more unexpected or creative responses.
- Top P: Similar to temperature, this controls randomness but uses a different method. Lowering Top P will narrow the model’s token selection to likelier tokens. Increasing Top P will let the model choose from tokens with both high and low likelihood.
- Stop sequence: Make the model end its response at a desired point. The model response will end before the specified sequence, so it won’t contain the stop sequence text. For ChatGPT, using <|im_end|> ensures that the model response doesn’t generate a follow-up user query.
- Frequency penalty: Reduce the chance of repeating a token proportionally based on how often it has appeared in the text so far. This decreases the likelihood of repeating the exact same text in a response.
- Presence penalty: Reduce the chance of repeating any token that has appeared in the text at all so far. This increases the likelihood of introducing new topics in a response.
Integration with Blob Storage Account (ADLS Gen II):
The integration of Blob Storage into the Chat Playground is not just a technical step; it’s a strategic move to infuse dynamic data interactions into our conversational AI application:
I. Creation of the dummy Blob-File
- Set up a ADLS (Azure Data Lake Storage) Gen II account.
- Add a Blob file to the Blob container, here we created a dummy travel itinerary file called ‘Itinerary for John Doe.docx’.
This dummy-file has the following content:
II. Adding a ‘Data Source’ in the Chat Playground of the Azure AI Studio
- Utilize “Add your Data” Function, the “Add your Data” function is the bridge that connects our Blob Chat application to the dummy travel itinerary data within the Blob container. Understand the intricacies of utilizing this function, which unlocks the potential of dynamic and context-aware conversations.
2. As shown in the image above, select the Data Source which is Blob Storage as per our current workflow and configure all the required parameters.
3. Also, create a new Azure AI Search resource, where the index used for grounding will be created.
4. Enter the index name, that will be used to reference this data source. Also, select the frequency at which the indexer runs.
5. Add vector search to the search resource.
Info: What is vector search?
Vector search in Chat Playground utilizes vector representations of text to find semantically similar responses, enhancing the model’s ability to generate contextually relevant and coherent conversational outputs. It improves the natural language understanding by mapping sentences into a multi-dimensional space for efficient comparison.
6. Select an embedding model that you can create in ‘Deployment’ to use a vector model as part of your data.
Info: Here in the current workflow, I have selected ‘gpt-35-turbo’ model.
III. Data management to set up specific configurations for your data and how the model will respond to requests.
- Set up specific configurations for your data and how the model will respond to requests. Select the ‘Search type’ to use with your search index.
2. Review the configurations you set for your data → Save & Close
Workflow Demo
Once the above steps are successfully completed. Its time for the demo.
For Reference, here is once again the dummy blob-file for travel itineraries.
Chat Session from the Demo
Deploy an intelligent “Blob-Chat” Travel Application
Configure the settings to deploy a Web App
Chat Session from Web Application
Observations from the Workflow Demo
- Note that the Citations or source of information from dummy Blob-File.
- To my 3rd question there is no response, as earlier in the Advanced settings of the Assistant setup we selected ‘Limit responses to your data content’.
- Also, there is ‘Chat History’ option on top that contains the chat-history of the Chat session. The chat history is stored in an Azure Cosmos DB, which is enabled from the configuration settings of Deploying the Web App.
Conclusion
As we navigate through the expansive landscape of Azure Open AI Studio, from Playgrounds to Search Services and the Chat Playground, the depth and breadth of its capabilities become apparent. The integration with Blob Storage elevates our conversational AI application to new heights, infusing it with the intelligence needed to comprehend and respond to dynamic user queries. Azure Open AI Studio isn’t just a platform; it’s a journey into the future of AI-powered applications, and by understanding its intricacies, developers can sculpt intelligent, responsive, and user-centric experiences.
This comprehensive exploration serves as a guide for those looking to harness the full potential of Azure Open AI Studio, offering insights and practical steps to navigate its features seamlessly. As you embark on your journey into the realm of conversational AI, armed with the knowledge imparted in this article, you’re well-equipped to craft applications that not only respond to user queries but do so with a level of intelligence and adaptability that defines the future of AI.
Future Work
While this article has delved into the powerful capabilities of Azure Open AI Studio within the Azure portal, the journey doesn’t end here. In the next article, we will explore the deployment of a Python application for the Blob Chat Travel Application, showcasing the seamless integration of Azure AI models into real-world applications. Stay tuned for more examples and practical insights into the ever-expanding landscape of Azure Open AI. The future holds exciting possibilities as we continue to unlock the full potential of AI in diverse applications.
