Member-only story

Evaluating search relevance on-demand with crowdsourcing

5 insider tips for crowdsourcing search relevance evaluations

Magdalena Konkiewicz
Towards Data Science
6 min readMar 29, 2022

--

Photo by the author

Introduction

One of the most essential e-commerce tasks today is search relevance evaluation. Your online marketplace relies on search algorithms to improve the customer experience but evaluating search data is challenging.

In this article, you will learn some insider tips on how to obtain consistently accurate human-sourced labels for search relevance projects.

What is search relevance evaluation?

Search relevance evaluation compares specific search results with the user’s search query to see if they match. In e-commerce, it’s used to improve search page results, select relevant ads, suggest accessories for customers to buy, etc.

Photo by the author

Labeled relevance datasets are used to verify the results of pre-trained ML algorithms and provide data for training new versions of the models.

Additionally, they can be used to test new features or different approaches for the search on a large audience without deploying to production or affecting the…

--

--

Towards Data Science
Towards Data Science

Published in Towards Data Science

Your home for data science and AI. The world’s leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.

Magdalena Konkiewicz
Magdalena Konkiewicz

Written by Magdalena Konkiewicz

Data Scientist, NLP, ML practitioner, and educator. Blogging from Medium and aboutdatablog.com. Support my writing: https://medium.com/@konkiewicz.m/membership

No responses yet