Vector Search at Trendyol Search Core

In this article, as Trendyol Search Core team, I will talk about technical and practical information about Search with Vectors, how we implemented it with Elasticsearch, its impact on our system, and what we can do in the future.

Introduction

We will give information about the technologies and experiences we used while implementing Vector Search, and we will give examples of why we chose Elasticsearch as the database we use for Vector Search and how we applied it to it. And also we will mention how we can use Vector Search more actively in the future.

The reason we added the Vector Search feature to the Search Domain is to show more personalized results to users or searches made by semantic search. This feature will allow users to see the products they want to reach quickly in higher positions when they search.

Let’s get started!

What is the Vector Search?

Vector search is a technique that represents the properties of data items as numerical vectors and performs similarity-based search operations through these vectors. Each item in the dataset is represented by vectors specifying its properties, allowing users to quickly find items with the most similar vectors given a query vector. This method enables efficient search and analysis of large datasets.

The main point we need to focus on is the similarity calculation of vector embeddings for two different models as you can see in the picture below.

Vector Search Databases

In Vector Search Database selection, the purpose for which we will use vectors and the domain you will implement are important, the other two points we should pay attention to in the search domain should be query time and indexing time.

As far as we examined in Elasticsearch’s documentation, it meets the current phases and the technical requirements that may come in the future phases. At the same time, it would have been more advantageous to choose a source where we had the know-how and would not need an additional external connection. So we decided to do tests with Elasticsearch.

You can examine the comparisons of Vector Search Databases by clicking on the link.

Elasticsearch Testing

In Elasticsearch, which is used as a Search Engine, filtering is done according to certain logic and sorting is done according to scores. With the database we chose, as-is query (score calculation, filtering, etc.) should work and we should be able to implement Vector Search. We realized that we can do this on Elasticsearch with the Rescore Query feature, which we will examine in detail in the following topics.

By creating a sample index with the PoC we did, we made it clear that we can use Elasticsearch as a result of indexing vector embeddings and querying with sample queries, profiling query performances, and extracting real usage scenarios.

Vector Search with Elasticsearch

Let’s examine the general terms and Elasticsearch features we will use in our mapping.

Dimension is the number of each argument of a vector in space. We have to specify this number in our Elasticsearch mapping and we can store vectors on Elasticsearch that are equal to what we specify, if they are not equal we get an error from Elasticsearch.

Note: For a document that does not have a vector embedding, you should determine a strategy and ensure that the vector is stored in that area and you should not negatively affect the search experience.

Dense Vector Type is a vector type that stores vector embeddings in float type by default. We chose this type because it can store zero values, it can be used with the kNN feature that we might use in future phases and it fits our vector embedding type. However, you can also examine the other vector type that Elasticsearch provides, Sparse Vector.

Note: Before Elasticsearch 8.0 the index property is not included in the mapping, in later versions it is used as default true. If you are going to use index true (for example with kNN), you can check Elasticsearch’s index_options property.

Similarity calculates the reflection between two items and the document score by applying the formula of the type we have chosen. When choosing the Similarity type, it is recommended to choose a function according to the model being trained, for us it was Cosine Similarity. It is explained in the Elasticsearch document as follows:

The cosineSimilarity function calculates the measure of cosine similarity between a given query vector and document vectors.

Script Score Query can be used with query and script fields. The vector embeddings on all documents filtered with query (the query part in the example and the reason for the match_all query in it) and the query vector embedding is used to apply cosine similarity for similarity calculation (the script part in the example).

Note: By using parameters in the source of the script we prepared, we prevented it from compiling on every incoming request and getting stuck in the script limit.

When a search request arrives, if the request is suitable for vector search, Rescore Query is built into our application and sent to Elasticsearch.

Rescore Query, by filtering the documents with query, documents up to the window_size value are obtained and the relevancy score is calculated. Additional relevancy score is calculated according to the logic running in the Rescore query and the final relevancy score is calculated according to the strategies we specify by filtering the documents up to the size value.

Note: The sortfeature cannot work in Rescore Query, it throws an error.

Let’s look at the theoretical details of Elasticsearch examples!

Mapping: The mapping definition of the vector embeddings that we will store in our documents on Elasticsearch will look like below.

{
  "mappings": {
    "properties": {
      "vector_field": {
        "type": "dense_vector",
        "dims": 3
      }
    }
  }
}

Document: An example of vector embedding stored on the document is given below.

    "vector_field" : [
      -1.502543577960682,
      0.47163141780930107,
      2.506715811684738
    ]

Query: As we explained in the Rescore query, first the query field is run and filtered and the vector_field vector embeddings on 100 documents and the query_vector_field vector embedding in the query are converged with linear scan, and the first 10 documents are returned.

{
  "from": 0,
  "size": 10,
  "query": {
    ...
  },
  "rescore": [
    {
      "window_size": 100,
      "query": {
        "rescore_query": {
          "script_score": {
            "query": {
              "match_all": {
                "boost": 1
              }
            },
            "script": {
              "source": "cosineSimilarity(params.query_vector,params.query_vector_field)",
              "lang": "painless",
              "params": {
                "query_vector": [
                  -1.502543577960682,
                  0.47163141780930107,
                  2.506715811684738
                ],
                "query_vector_field": "vector_field"
              }
            },
            "boost": 1
          }
        },
        "query_weight": 1,
        "rescore_query_weight": 1,
        "score_mode": "total"
      }
    }
  ]
}

Note: With the Query and Rescore weight values, you can specify which query you want to put more weight on the relevancy score calculation and in which mode these two scores will be calculated. The values specified in the example are set by default.

The trained model and the resulting vector embeddings are important for the impact of our search results with Vector Search. We wanted to talk about how we technically implement it in the search domain.

Impact on the System

The importance of the impact on the system depends on the number of documents on which we will keep a vector. If you need to add a vector to each document, you should check the resources of your system by calculating the data size to be added. You can calculate the type (e.g. float64), and dimension size (e.g. 512) of a vector we need to store based on the total number of documents and how the database you will use stores vector embeddings.

Result

Our team always aims to see possible problems that may occur by running a load test after adding the new features and before starting to use the features in the production environment at regular intervals.

In the tests we did with this feature, we did not see any negative effects by applying the recommendations we gave, such as paying attention to the use of scripts, scaling the system, not having too large a number of products accessed with window size, and our project started to be used in the production environment.

What could be in the future?

With Rescore Query, as-is and vector search relevancy score calculations can be done with kNN search, which is also supported by Elasticsearch, only with vector embeddings. You can review the document for more details.

Conclusion

In this article, I wanted to talk about our experiences and technical details about how we implemented Vector Search as Search Core team.

Thanks for reading the article, you can contact me for your feedback and questions. See you in my next articles!

#ComeToTrendyol

If you want to make a new start in your career and join us, you can access the open positions from the link below.

Trendyol jobs