At Thumbtack, we’re focused on becoming the best platform to fix, maintain, and improve your home. We achieve this by connecting customers to the right home service professionals for the task at hand. To deliver a great experience to both our customers and professionals, our engineering teams make sure that the algorithms matching customers to professionals are reliable even though they tweak and refine them every day. The simulator is a command line tool we built that helps us detect ranking regressions and maintain ranking quality.

Our ranking algorithm takes in a customer’s search and outputs a ranked list of professionals to show the customer (details in this post). It also returns extra data to show in the UI, such as relevant review snippets or tags to highlight professionals that stand out. Because our ranking algorithm is complex, developers can introduce regressions when making changes. Regressions cost Thumbtack revenue, give customers poor experiences, and hurt professionals’ lead prospects. Hence, it’s very important to us to avoid them as much as possible using tools like the simulator. …

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store