Semantic Search: Stop Looking for Words, Search for Meaning

AI generated image from Freepik

Introduction Semantic Search

In the realm of smart search technologies, the journey continues with an important exploration into the transformative capabilities of Semantic Search. The shift is profound — no longer confined to the limitations of static keywords, we embark on a quest to stop merely looking for words and, instead, delve into the profound world of searching for meaning.

This blog is part of a series on Smart Search technology. This is the second part. Read the introduction to get an overview of smart search using Large Language Models and other types of search, before diving into this article about Semantic Search.

Semantic Search Explained

Semantic Search represents a paradigm shift in how we approach information retrieval. Instead of fixating on individual words, this innovative approach focuses on capturing the essence of meaning and semantic relationships. It’s a departure from the traditional methods, offering a more nuanced understanding of user intent and contextual relevance.

To use Semantic Search, we convert the content to vectors using a embedding model, a specific type of Large Language Model (LLM). The content is converted into vectors, which are called embeddings. This is also why semantic search is sometimes referred to as vector search. When a user sends a search query, a set of keywords, or a complete sentence/question, the embedding model is also used to convert it to an embedding. Semantic Search calculates the distance between the embedding of the user's input and the embeddings of the documents. The search results consist of the documents that are semantically closest to the user’s input, meaning they have the nearest embeddings. This method of search is known as Semantic Search because it matches the meaning of the user’s input to the content of the documents.

Multi-modal input and content

Most often, when mentioning content, people refer to text. This type of content consists of multiple lines, paragraphs, pages, or chapters. One can choose to break up the content into chunks to embed these chunks instead of the complete text. This is also important because of the length of input the LLM can handle, but also because your content often consists of multiple topics. Then it is best practice to do chunk optimization, where you select the right chunk size, for the best search result for your content.

Content of other types, like video, audio, images, or other formats of data, can also be converted to vectors, put into a vector database, and perform semantic search on these vectors. A vector database can contain vectors of all types of content in one vector space. This means that one can input text and for example, get out images that are semantically close to this input text. Before semantic search was popular, one had to analyze and annotate this content and filter to get the right content. Now, also both techniques can be combined by also embedding metadata or using the metadata to first filter a specific part of content and then search for semantic similar embeddings.

Or think about a tool like ChatGPT, where users can input an image or a voice command and find images that are similar. For this article, we will not go further into this part of semantic search. The most promising, easy and valuable use cases right now, are about making text better searchable. Also, when you productionized semantic search on text, one can later on get into other content types. But if you want to dive right in, go ahead! We are here to help.

Beyond Keywords

The power of Semantic Search lies in its ability to navigate the intricacies beyond keywords. It goes beyond the surface, diving into the layers of meaning embedded in language. As we explore this NextGen technology, we’ll uncover how it enhances precision, captures subtle nuances, and elevates the overall search experience.

In the world of Semantic Search, the search isn’t just about finding relevant content; it’s about discovering meaning within the context of the user’s query. Imagine a search experience where the results are accurate and resonate with the deeper intent behind the words — a journey towards a more refined and sophisticated search encounter.

So, when you are thinking about implementing semantic search think about at least the following:

• Decide where your data is protected enough. Are you choosing an API in the public cloud or do you want to run a model on your hardware? Think about data security and privacy, when designing your AI system.
• Closed-source vs open-source LLMs. Do you know on which data the LLM is trained? How this model is trained and how humans were involved in evaluating and training this model? It is still uncertain how some popular closed-source models are trained and on what data. This continues to cause discussions about intellectual property and copyright.
• Pick the right LLM for your use case, language, and domain. For instance, not all language models support the Dutch language. Is an LLM not working for your use case, language, or domain? Consider picking another model or fine-tuning the existing model. Look into open-source models on Hugging Face and find the right model for your task and language. Also, look at the license of the model.
• Think about the right chunk strategy. Picking the chunk size that is optimal for your use case is key. Evaluating performance for your AI system is important to see the difference between different chunk sizes.
• There are a lot of vector databases out there, and existing databases that added vector and vector search support. Determine what vector database fits your use cases. Have a look at this Vector DB Comparison.

Drawbacks

When using Semantic Search to find the right answer, there are a few drawbacks that you should be aware of.

• Users are used to write short keywords list to find the most relevant content. They are not used and maybe not motivated to completely type out their question as a complete sentence. Example, users search ‘restaurant french’ instead of ‘What restaurants are close by that have French food?’.
• Semantic Search can always return results, because it just searches for the most similar embeddings and returns the result. So there is not a ‘No results found’-message to be shown.

In order to address both drawbacks, we can also look into hybrid search. Hybrid search combines the best of keyword search and semantic search. Both techniques are executed and the results of both are ranked to show to the user.

Another way to address the first drawback is to look into query expansion. Query expansion analyzes the user input and looks for extra details and context to add to the query to find better results. This can be enhanced by using entity recognition to find occurrences of certain entities and concepts. The extra context can be added to the query in several way, for instance combining a Large Language Model (LLM) and a Knowledge Graph (KG).

In order to really grasp all the possibilities of semantic search, I would suggest reading this article about modular RAG.

Conclusion

As we wrap up this exploration into Semantic Search, we’ve seen how it transforms the search experience. It’s not about looking for words; it’s about searching for meaning. Join us in the next blog as we dive into a new topic within Smart Search to further enhance the depth of our smart search capabilities and understanding.

Stay tuned for the third blog in this series!