The LLM Revolution: How AI is Reshaping Search

Niraj Namjoshi
Mozilla Firefox Club VIT Vellore
6 min readAug 8, 2023

You may have recently heard a lot of talk about LLMs and how they are changing the very foundations of the internet.

In order to optimize search results and make them more accessible and user-friendly, LLMs are now essential, but first, let’s define what an LLM is.

LLM, which stands for large language model, is a word used in the field of artificial intelligence to refer to models that are very large and have been trained on billions of parameters for either general or specialized purposes. A good example of such models is BARD, GPT, and LLAMA, which are used by millions of users every day.

One of the primary benefits of such LLMs is their ability to take a wide range of user inputs and offer accurate and relevant responses that are tailored to the user’s request. As a result, such models are increasingly being integrated into search engines in order to optimize results and provide more user-friendly and organic material.

HERE IS AN EXAMPLE:

Imagine you are a travel enthusiast planning a weekend getaway to explore the beautiful landscapes of a specific national park. You want to find the best camping spots within the park, but when you search online for “Top Camping Spots in XYZ National Park,” you are overwhelmed by the multitude of blog posts and outdated information.

This time, you decide to use an AI-powered travel recommendation tool called “WanderWise.” You tell the tool that you want to find the top camping spots inside XYZ National Park with specific criteria like scenic views, proximity to water sources, and availability of facilities.

WanderWise utilizes its AI capabilities to analyze real-time data, user reviews, and official park information. Within seconds, it presents you with a curated list of the five best camping spots that match your preferences. One of the spots stands out, located by a picturesque lake, with rave reviews from recent visitors.

Excited by the prospect of camping by the lake, you decide to inquire further and ask WanderWise about the availability of permits for that particular spot. WanderWise instantly checks the park’s permit system and informs you that there are a few spots left for your desired dates.

You are thrilled with the efficiency of the AI tool and decide to book the permit right away through WanderWise, skipping the traditional process of visiting the national park’s website.

This leads to the question: will AI-driven travel recommendation platforms like WanderWise impact web traffic for national parks and camping websites? Will traditional SEO strategies still be as effective when users increasingly rely on AI tools to gather information and make decisions?

How LLMs respond to search queries:

When you converse with models like ChatGPT and Bard, the odds are good that they are searching their library of training data for information on the object you are requesting.

Similar to the probabilistic data structure of a Bloom filter, these models get their output from a learned manifold based on training data.

In order to include as much input as feasible, the model builds a continuous manifold based on the training data during training. Its responses during query time are interpolations within this learned manifold.

These trained LLMs can then function by taking the information you submit, parsing and encoding it into a vector representation, then scraping the web and choosing pertinent data from a variety of sources such as articles, posts, user comments, user reactions, etc. based on the vector representation.

The gathered information is then reorganized to make it easier to read and more logical.

LLMs are also well renowned for their abilities in sentient analysis; therefore, it is suitable to employ them to filter out content for customized responses and recommendations.

While this method allows us to provide concise answers relevant to user queries, it also has some disadvantages:

  • Bias: LLMs are trained on massive datasets of text, and these datasets can be biased. This means that LLMs can generate biased search results.
  • Accuracy: LLMs are not always accurate and cannot guarantee that the response is factual.
  • Interpretability: LLMs are often difficult to interpret. This means that it can be difficult to understand how LLMs generate search results. This can make it difficult to trust LLM-powered search engines.
  • Personalization: LLMs can be personalized to the user’s interests and preferences. However, this can also lead to echo chambers, where users are only exposed to information that confirms their existing beliefs.
A OVERVIEW OF THE ARCHITECTURE OF A LLM SEARCH ENGINE (LANGCHAIN + VECTARA)

How LLMs Will Affect User Traffic and How Websites Can Optimise to Stay on Top:

As AI-powered tools become more sophisticated and user-friendly, they have the potential to significantly impact web traffic patterns and SEO(Search Engine Optimisation) strategies. Websites and businesses may need to adapt by optimizing for AI-driven queries and ensuring their information is up-to-date and easily accessible through such tools. The landscape of how people access information and make decisions is evolving, and traditional search approaches may need to embrace the AI revolution to remain relevant in the digital age.

So, how do you adapt your SEO tactics for LLM-powered search engines?

  • Optimize for semantic search: LLMs are able to understand the meaning behind search queries, so it is important to optimize your content for semantic search. This means using natural language and long-tail keywords, as well as incorporating synonyms and related terms.
  • Create high-quality content: Traditional search engines relied heavily on keyword matching. But LLMs, with their advanced understanding of natural language, context, and semantics, take search to a new level. They aim to grasp the user’s intent, not just their keywords, making SEO less about keyword stuffing and more about meaningful, valuable content.
  • Structure your content well: LLMs use the structure of your content to understand what it is about. Make sure your content is well organized with clear headings and subheadings. You should also use schema markup to help LLMs understand the meaning of your content. For example, you could use schema markup to indicate that your content is a recipe or a health article.
  • Build links to your content: Links are still an important ranking factor for SEO, so it is important to build links to your content from high-quality websites. You can do this by guest blogging, submitting your content to directories, and participating in online communities.

By following these strategies, websites can improve their ranking in LLM-powered search engines, staying ahead of the curve in the digital age.

CONCLUSION

The emergence of LLMs is ushering in a new era of internet interaction, fundamentally changing the way users access information and make decisions. These powerful AI-driven tools excel at understanding user intent and delivering relevant and personalized responses. As LLMs become increasingly integrated into search engines and recommendation systems, websites and businesses must adapt their SEO strategies to optimize for semantic search, produce high-quality content, structure information effectively, and build authoritative links.

LLMs are already having a major impact on the way we interact with the internet. For example, they are being used to power more natural and conversational search experiences, such as those offered by Google’s BERT and LAMDA models. LLMs are also being used to generate more personalized and relevant search results, which can help users find the information they need more quickly and easily.

In addition to their impact on search, LLMs are also being used to automate tasks, such as writing blog posts, creating social media content, and generating customer support responses. This can free up human workers to focus on more creative and strategic tasks, while also improving the quality and consistency of the output.

The ongoing evolution of LLMs promises to reshape the internet landscape for the better. However, it is important to be aware of the potential challenges and pitfalls, such as biases and interpretability issues. As LLMs become more integrated into our lives, it is important to work to mitigate these risks and ensure that they are used in a responsible and ethical way.

--

--