Member-only story
Evaluating search relevance on-demand with crowdsourcing
5 insider tips for crowdsourcing search relevance evaluations
Introduction
One of the most essential e-commerce tasks today is search relevance evaluation. Your online marketplace relies on search algorithms to improve the customer experience but evaluating search data is challenging.
In this article, you will learn some insider tips on how to obtain consistently accurate human-sourced labels for search relevance projects.
What is search relevance evaluation?
Search relevance evaluation compares specific search results with the user’s search query to see if they match. In e-commerce, it’s used to improve search page results, select relevant ads, suggest accessories for customers to buy, etc.
Labeled relevance datasets are used to verify the results of pre-trained ML algorithms and provide data for training new versions of the models.
Additionally, they can be used to test new features or different approaches for the search on a large audience without deploying to production or affecting the…