Why CAI is the Future of Search
This story is a rough-draft. Check back later for the fully-polished story.
I argue that the inevitable future of most — if not all — search engine products will take the form of conversational artificial intelligence (CAI).
I’m not the only one who believes this.
All consumer-facing software will one day have a dialogue interface, and this is the next vital leap in the evolution of search engines.
Tao Yu, 2021 PhD Dissertation, Yale University
First, what is CAI? One way to describe CAI is by listing the main processes that allow CAI to work. They are the following: (1) language understanding, (2) knowledge and context management, (3) dialog management, (4) language generation.
Many search experiences trip over the way the search engine cannot understand complete sentences. As Google recently explained, search engines become more useful the better they understanding natural language and the less their users have to guess at how to write in “keyword-ese”. Search will never be as good as it can be until it can understand any natural language sentence.
Many searches are a dialog between user and search engine already. Why not make that dialog more natural and effective? For example, say you go to Google to find out if the best international football player suffered any major injuries in his last game. But you do not know who the best football player is when you start. So you — being a well-trained search engine user — mentally deconstructs your question into a series of queries. First query: find out who the best football player is. Second query (if you are lucky): find out when he played last. Third query: find any news you can about that player having injuries during that game. It’s a dialog, except that for many of those steps you will probably have to rephrase your queries to match the wording that appears on whatever pages you are trying to find — before you actually find them. Plus, you will be expected to remember all the intermediate information to pass on from one query to the next. Wouldn’t it be nice if the search engine itself kept track of that information for you? In fact, wouldn’t it be nice if you could simply ask a question like “Did the best world football player have any major injuries in his last game?” And if the search engine doesn’t tell you the player’s name in its response, you could carry on a conversation about it by asking follow-up questions like “What is his name?”
I see that Adobe has a similar vision.
Almost all searches return results that the user does not need to read and will not read. If the search engine could ask the user questions specifically designed to help the user narrow down the search results to what that user needs, there would be no reason for the search engine to try to off-load some of its work to the user by dumping a huge pile of search engine results on that user and expecting him/her to sift through them. In some recent interfaces such as voice, it is not really possible to dump those results anyway.
