Analysis of Bing AI
At the Version 1 Innovation Labs we have been researching, testing, and developing use cases for the latest breakthroughs in AI. We recently wrote about ChatGPT and GPT-3 and its peers.
Now, the latest AI on the block is Bing AI, a search engine that uses Artificial Intelligence (AI) to provide better search results, thorough responses, create a new chat experience, and the ability to create content like nothing before it.
Bing AI has two features: search and chat. The chat feature engages users in a conversation, asking questions, responding, and having several back and forths Much like when you converse with a human being!
It also offers search results for common queries like sports scores, market prices, the weather, etc, this comes with a new sidebar that displays in-dept information if you need. Bing AI analyses search results from several websites to identify and condense the solution you’re looking for.
What makes Bing AI stand out, and pose a threat to stalwarts like Google and now ChatGPT3, is the way it leverages GPT4 making it faster, more accurate, more powerful.
Bing AI is capable of providing up to date information as it combines results from Bing search and AI models output, hence overcoming one of the main ChatGPT’s limitations.
Bing AI is uniquely able to help users search for information using keywords or questions, and then provides output in a text format.
Try It Out For Yourself
Bing AI is available for preview, you can sign up here. Users can make an information request, and Bing AI will return instant results by collecting information from the web as well as using outputs from the AI models integrated within the search engine.
Key details about Bing AI:
· Microsoft developed a proprietary technology known as Prometheus.
· This is a unique model that combines the fresh and comprehensive Bing index, response rankings, and results with the creative reasoning capabilities of the most advanced OpenAI GPT model.
· Prometheus leverages the power of Bing and GPT to repeatedly generate a series of internal queries via a component called the Bing Orchestrator to provide accurate and comprehensive responses to user queries within the specific context of the conversation.
· This response generated by Prometheus is called a chat response.
· Selecting relevant internal queries and leveraging respective Bing search results is a key component of Prometheus. This provides up-to-date information relevant to the model, helps answer topical questions, and reduces inaccuracies.
The following diagram shows how Prometheus works at a high level:
An example:
Article Writing Based on US unemployment rates
A dataset (from kaggle.com ) containing US unemployment rates was fed to Bing AI chat and asked to draft an article.
It can identify data and statistical values and insights based on the input provided. Summary looks approximately correct. It gave results based on the data provided and also took results from the web and combined both results in its answers.
Same dataset was fed to ChatGPT and asked to draft an article
The ChatGPT summary is fairly plain. The Bing AI summary is rich in information, annotation, and facts.
Limitations of Bing AI :
One of the limitations of Bing AI is that we can currently input only 2000 characters in the query. Additionally, the number of chats per day for the user is set to 120. It is important to note that this specification is subject to change rapidly, and the current limit may not remain the same in the future.
Bing AI also limits users to 10 queries per session, after which it prompts the user to start a new topic. Bing says this is done to avoid a lengthy back and forth chat.
Also, when a user hits the limit of 10 per session (means 10 queries after that it will ask to start over), Bing prompts the user to start a new topic to avoid back-and-forth long chat sessions. Microsoft argues that this is not a limitation, but rather that the chatbot gets confused with lengthy, non focussed queries, due to which it could go off the rails and discuss violence, declare love, and insists it was right when it was wrong!
It could be a limitation or could be not, as Microsoft saying that if the conversation is long, chatbot gets confused due to which it could go off the rails and discuss violence, declare love, and insist that it was right when it was wrong. It is unclear if Microsoft will remove this limit or not.
Bing AI vs ChatGPT
One of the limitations of Large Language Models (LLMs) like GPT is that they are trained on a finite data set. In particular, LLMs often study data up to a point in time. So, while it’s useful for some use cases, it’s not an option for content based on real-time data. Microsoft Bing AI has overcome this limitation with Prometheus, which uses Bing data and GPT to generate answers quickly while using up-to-date information.
An example comparing results from both Bing AI and ChatGPT (powered by LLMs):
Query: Write a 500 words report summarising today’s stock market?
We can see below that Bing AI’s answer is correct as it is searching for real time information. For validating the results, we have google searched results below
Bing AI Currently in Pole Position!
From this research, we find that Bing AI tool is not just a search engine, but also a chat based model and can provide answers related to queries. It can also can remember past messages for a period of time and provide contextual results.
The main feature of Bing AI which make it unique and powerful compared to other search engines is the integration of web search with an LLM model like GPT to provide relevant and up to date results to the queries of the user. Due to this feature, it looks more powerful than ChatGPT.
We can see a viable use case in businesses, as we know Bing integrated AI models to make its search engine more powerful. We can leverage GPT models in internal search tools to make search tool more accurate and faster.
About the author:
Hemant Vikani is a Data Scientist at the Version 1 Innovation Labs.
