ChatGPT and the future of search
Abstract: ChatGPT offers a more user-friendly search experience that relies less on traditional search engine optimization (SEO) methods and more on contextual prediction, generating fresh and natural outputs in response to prompts that range from simple to complex. LLM-based AI has the potential to change the way we think about search and its ethical implications, opening up new business opportunities and innovative ways of combining AI and traditional search engines.
The age of AI has arrived! ChatGPT marks a success story about generative artificial intelligence (AI) to-date. The chatbot AI tool that took 5 days to reach 1 million users, has been touted as the next biggest thing in the tech history — an innovation that “will change the world” — by Microsoft’s Bill Gates. Despite its current limitations, and plenty of ethical concerns and hype talks about AI chatbots, the nascent AI tool may have changed what we know about “search.”
ChatGPT is a large language model-based AI, or LLM AI. This means that the output from a searcher’s prompt is textual as in natural language. It relies on neural network — machine-learning algorithms that mimic the functioning of neurons in the human brain — that has been trained on 300 billion words using textual content scrapped from the web. Currently, the data fed into ChatGPT is limited up to 2021. It also relies on human validators who ranked answers generated by the neural network which are then fed into the AI to generate greater precision search results. ChatGPT uses contextual prediction that attempts to offer answers based on the context of the prompts entered.
Internet searches used to be driven by tech giants like Google and Microsoft, with their proprietary algorithms that pull together the most relevant web links and display them in ranked order. Google’s search algorithm has become multibillion-dollar industry, built on search engine optimization and paid search. Using certain “key words” and paying to be found help optimize a web page’s discoverability.
ChatGPT is a classic example of how new entrants disrupt (or even potentially replace) incumbents if the incumbents are too slow to react. To start with, even our language has shifted in recent months from “search(ing)” to “prompt(ing).” How will Large Language-based Model (LLM) AI such as ChatGPT and others disrupt or at least change the search industry? I suggest at least three changes.
First, at least initially, search is going to be less vulnerable to control — au revoir to the conventional SEO method — because LLM AI will churn out or pull together textual content without relying on optimized or paid search keywords. Rather than giving the searcher a long list of web links to click and read — which can be cognitively demanding — LLM-based AI’s smart neural networks create new combinations of content as fresh output.
Some scientists even claimed that “ChatGPT can do things that it is not trained to do.” Simply put, burden on the searcher to click and read the searched the content will be less.
It may not be surprising to see a shift toward AI-engine optimization, where clients pay to be found in AI chatbot results. Let’s calls this “AI-SEO”. However, there is a lot of credibility when prompt results are not based on paid outcomes although this is another conversation altogether.
Second, ChatGPT makes search feel more natural. Instead of feeling overwhelmed by the technicalities of doing search — questions like “should I use quotation marks (“What is generative AI?”) or Boolean codes (AND or OR) or how long should my search string be — a user can simply type a prompt in LLM-based AI machines in natural language (e.g., “tell me how to make a pizza for people with a keto diet”, “how to minimize risks in insurance products”, “What content should I include for a 12-week statistics course?”). The prompt results is delivered in very natural in the form of several paragraph long sentences to a single page long. This is more user-friendly and possible because LLM-based AI is built upon natural language processing (NLP) method rather than using content ranking a la Google’s current search algorithm.
Third, related to the above, search is going to be more creative in the way that the content that the searcher generates will very much depend on the prompts entered. There are two sides to this: whether the prompter is a novice or expert user, and whether the prompter is a domain expert or lay person.
A novice user of ChatGPT will likely use simple (very basic) prompts and accordingly produce very basic output that that often does not show the advantage of LLM-based AI. An expert/creative user will use a bunch of (more complex, iterative) prompt strategies that produce more refined, non-obvious and sophisticated results.
Furthermore, being a domain expert will further shape the prompt process and output. An accountant or a surgeon or lawyer or engineer or professor can craft deeper and contextually aware prompts that yield more refined results than a lay person who asks questions on accounting, medical, legal, to engineering questions. Somehow, to a novice user who is a non-domain expert, LLM-based AI will look and feel a little dumb, but to expert users with domain expertise it will be like having a dialogue with Leonardo Da Vinci or Albert Einstein.
The rumination above seems to suggest a black-and-white state of search. But hybrid search model may be what’s coming next — and business opportunities around it. Examples include searchers who use a mix of tools. They might use ChatGPT to grab a summary on a single page and then use Google to search for more precise web links to a particular topic (e.g., a company’s product page, a technical info of a technology, or mathematical formulas).
Another hybrid model is a new form of search engine that combines AI and Google-style search results that presents both the summary and weblinks side by side. This can be powered by SEO and AI-SEO simultaneously to serve web owners who are willing to pay to be found easily online. This could be the next evolution for search engines like Microsoft’ Bing AI or Google’s Bard, and countless of AI new entrants. Perhaps the answer lies in science fiction, that could well inspire the future of ‘prompt’.
This article was originally published in an op-ed in SCMP on 02/03/2023. The author is an associate professor at the City University of Hong Kong where he conducts research on emerging technologies from the management and organization perspective.
