Staying Ahead of the Curve Before and After the Generative AI Disruption
Being a leading player in the industry for several years, Version 1 constantly pushes the boundaries of innovation to stay ahead of the competition. We leverage cutting-edge technologies and have a deep understanding of customer needs to create a unique value proposition that differentiates us from other players in the market. With the rise of Generative AI, we have adapted quickly to stay ahead of the curve.
Let us explore how Version 1 was at the forefront of innovation for more than two years before the Generative AI disruption and how we continue to adapt to the changing market landscape.
Our Suite of Innovative Products
Version 1’s Innovation Labs collaborated with customers to build the ‘Smart Action Suite’; a collection of enterprise-ready productivity apps that address common challenges faced by organizations. Some of the products in the Suite are:
· Smart Bot: A chatbot that uses Azure Cognitive Services to offer personalized customer service and support.
· Smart Text: Text analytics solution that uses AI to provide deep insights into documents. The key features include sentiment analysis, named entity recognition, regular expression extraction, intelligent summarization, and topic modelling.
· Smart Search: Content search and discovery tool that uses advanced Natural Language Processing techniques to interpret and process large amounts of data. It allows users to semantically search for information within their corpus of documents, eliminating the need for manual analysis of large documents, and increasing revenue and employee productivity.
These products have been well-received by customers, as they offer unique solutions to common organizational problems. The suite’s innovative features and capabilities have made it an effective tool for businesses looking to improve their productivity and gain deep insights into their data.
The Evolution of Smart Text and Smart Search and why is it important?
The journey of Smart Text and Smart Search began in 2019 with the use of Spacy, a popular Python library which provided basic NLP functionalities such as tokenization, part-of-speech tagging, and named entity recognition. However, as the need for more advanced techniques arose, we moved on to using transformers, which allowed for more complex language tasks such as text classification, question-answering, and summarization. Transformers are based on a deep learning architecture that allows the model to learn from vast amounts of data, making it more accurate and efficient than traditional machine learning models.
We started on transformers long before the disruption caused by Open AI‘s Chat GPT Generative AI and we also leveraged the benefits of using Spark, a distributed computing framework that allowed for faster processing of large amounts of data. The Hugging Face models have also been instrumental in enhancing the capabilities of Smart Text and Smart Search. These pre-trained language models can be fine-tuned on specific tasks and have been shown to achieve state-of-the-art performance on various NLP benchmarks. This has allowed Smart Text to perform more complex language tasks with higher accuracy and efficiency. Thus, enabling organizations to extract valuable insights from unstructured text data, improving decision-making and increasing productivity.
With the boom of Generative AI, we were curious to find out where our Smart Search stands in comparison to other Search-systems in the market.
The Systems tested were:
- Smart Search combined with Open AI’s Completions API
- Open AI’s Embeddings API combined with Open AI’s Completions API
- Azure Cognitive Service — Custom Question Answering (built on top of Azure Cognitive Search)
Our research focused on comparing three different systems based on:
- Accuracy
- Response time
- Ability to handle answers that are spread across pages.
The study used a custom dataset of over a hundred documents. While each system demonstrated unique strengths and limitations, the Embeddings-based system achieved the highest accuracy rate (60% accuracy), while Azure’s Custom Question Answering and Smart Search-based system followed closely behind with a relatively small accuracy gap. Although the Custom Question Answering system performed well with straightforward queries, it faced difficulties with more complex and ambiguous questions. Similarly, the Smart Search-based system faced some challenges as it was primarily designed for short queries of up to three words and not intended to function as a question-answering system.
The study concludes that providing adequate context and training data is crucial to improve accuracy and relevance, and ongoing maintenance is necessary to ensure that the systems continue to provide accurate answers as new data becomes available. The research provides valuable insights for selecting the appropriate system for specific applications.
Even though Smart Search may not be as advanced as GPT-3 or the newer models, it still offers a range of valuable capabilities that make it an indispensable tool for organizations seeking to enhance their semantic search and text analytics capabilities. For instance, Smart Search can perform semantic searches of documents, saving organizations precious time and boosting their overall productivity. Smart Text, on the other hand, can provide in-depth insights into documents through text analytics, such as sentiment analysis and topic modelling. With these capabilities, organizations can unlock hidden value from their data and gain a deeper understanding of their customers, products, and markets.
One of the notable features of Smart Search and Smart Text is their flexibility. Both solutions can be deployed on-premises or in the cloud, providing businesses with the flexibility they need to choose the option that best suits their needs. In addition to this, both solutions are also designed to be cost-effective. They offer a competitive price point compared to similar technologies in the market, making them accessible to businesses of all sizes. Installation and implementation are hassle-free, along with a seamless and optimized user experience, enabling businesses to derive maximum value from their data cost-effectively and efficiently.
Our Investment in State-of-the-Art Technologies
We are committed to investing in modern technologies since we know the importance of continuous evolution and adaptation to stay ahead of the curve. The Version 1 Innovation Labs are constantly exploring ways to apply them to the company’s products and services.
One of the most significant disruptions in recent years has been the rise of Generative AI. Generative AI is a form of artificial intelligence that can generate content such as text, images, and videos, and it has the potential to transform the way businesses operate. However, this disruption also presents a challenge to companies that are not prepared to adapt to the changing market landscape.
At Version 1, we recognized the potential it has for disrupting the business model early on. Instead of being caught off guard, we were quick to adapt to the changing market landscape. We have done extensive research on all the models that have come out in the Generative AI space, whether it be Meta’s Llama, Microsoft’s Bing AI or Google’s Bard. We continue to investigate how we can better support our customers using these latest models.
For example, we are currently working on integrating Azure Cognitive Search and Azure Open AI, two powerful tools that can help our customers extract insights from large amounts of unstructured data. With these tools, businesses can automate their data analysis processes and make better-informed decisions based on the insights generated by Generative AI.
Conclusion
Version 1’s commitment to investing in innovative technologies and understanding its customers’ needs enabled it to create innovative products and services that provided unique value propositions to customers. Our quick response to the Generative AI disruption demonstrates our ability to adapt to the changing market landscape.
As technology continues to advance, it is essential for companies to invest in state-of-the-art technologies like Generative AI and adapt to the changing market landscape to stay ahead of the curve. At Version 1, we are committed to doing just that, and we will continue to explore new and innovative ways to apply advanced technologies to our products and services to better support our customers.To learn more about our range of products and services, reach out to Innovation Labs. Our team would be happy to provide you with further information and answer any questions you may have.
Find out more about the Version 1 Innovation Labs here.
About the Author:
Geethu Suresh is a Microsoft .NET Consultant here at Version 1.
