What is Semantic Search?

Source: iStock photo

If I were to do a traditional web search for “Astro”, one of the Salesforce mascots, I’d get some results that are useful. But I’d likely also get some unrelated results like:

• Astronomy
• Houston Astros baseball team
• American Society for Radiation Oncology (ASTRO)

That’s because search engines have historically been driven by keyword, or lexical, search. The search engine serves up relevant web pages by giving me the literal matches of the words I type into the search bar.

What is semantic search?

To understand semantic search let’s first start with what the word semantics means.

se·man·tics

/səˈman(t)iks/

noun

the meaning of a word, phrase, sentence, or text.

So semantic search means searching by meaning instead of searching by something else, like a keyword. If we take our example from above and utilize semantic search instead of lexical search, I might get back additional search results for other Salesforce characters like:

• Cloudy
• Codey
• Ruth

This is because semantic search uses meaning, or context, to find matches. These results might not match the literal search term I typed in, but they match the intent of my search. But how does this work? How does a search engine, or a machine for that matter, understand meaning?

How machines interpret meaning

For many years technologists have been using the methods like semantic HTML and intent tables to label and store data needed to generate more meaningful search results. But this, for the most part, this has required human interaction, because humans are the ones who understand the meaning behind the words we choose.

But in machine learning there is a concept called vector embeddings. Vector embeddings are an essential part of how new generative AI tools like ChatGPT work. Stay with me on this one, GPT stands for generative pre-trained transformer. The “transformer” part of GPT refers to the process of encoding and decoding things that humans can understand like language or visuals, into a format that a machine can understand.

One of the formats a machine can understand is a vector embedding. Vector embeddings are essentially lists of numbers that represents the original text, image, audio, or video. Now this is where it gets really cool.

As previously stated, a vector embedding is a series of numbers, but these aren’t just any numbers. They’re coordinates for how the original object (the text, image, audio etc) is represented by the system. If this is hard to visualize, imagine the old school dewey decimal system in the library where books with similar characteristics are grouped together by classification.

Source: iStock photo

You use this single set of numbers to find the location of the book you’re looking for. Coordinates on a map use two dimensions to locate your destination.

A similar mental model can be applied to get a basic understanding of vector embeddings. While our simple examples above use one or two dimensions, the vector representation used in natural language processing can have thousands of dimensions.

At a very high level, here’s how an object that a human can understand is transformed into a format that a machine can understand. The object is fed through an embedding model (such as T5 or BERT for example). This model does the heavy lifting of encoding our original object to an embedding (an array of numbers).

Diagram showing the process of creating an embedding

Then, using some pretty fancy mathematical functions like cosine similarity, these embeddings can be plotted on a vector (an arrow with a length and a direction, as pictured below). Objects with similar meaning get plotted near each other in vector space. As a result, a machine can use the angle and distance between vector embeddings as a proxy for the semantics (or the meaning) of a piece of text, image or audio.

Example of embeddings plotted on a vector (Source: Google — “Meet AI’s multitool: Vector embeddings”)

Conclusion

Back to our original scenario of searching for the Salesforce character Astro. If Astro was translated to a vector embedding, his coordinates would be in the same vector space as Codey, Cloudy and Ruth so our semantic search would return our entire crew of Salesforce characters.

With the rise of generative AI, understanding how semantic search works is important because it is going to transform the way we search for information. It will also change the way we interact with applications like Salesforce. In fact one of Salesforce’s newest AI pilot features, Einstein Search Answers, is powered by semantic search and vector embeddings. To explore more upcoming product innovations visit the Salesforce Roadmap Explorer on architect.salesforce.com.

For the press release on Salesforce’s new Vector Database built on Data Cloud visit: https://www.salesforce.com/news/press-releases/2023/12/14/unstructured-data-ai-search-einstein/

Watch “What is Semantic Search?” on YouTube

References

Towards More Intelligent Search: Deep Learning for Query Semantics
Semantic Search with Embeddings: Index Anything
A Comprehensive Overview of Transformer-Based Models: Encoders, Decoders, and More
What are Vector Embeddings
Meet AI’s Multitool: Vector Embeddings
Salesforce Roadmap Explorer