Sorts and Filters Suck

The traditional search interface, with its array of filters, facets, and sorting options is ubiquitous. It also sucks. An explanation might not have been required, but for a suspicion of mine. A suspicion that databases have spent the last few decades training us. They’ve been training us to think about search in ways that are easy for them to implement.

It’s time to remind our databases that they work for us.

To illustrate, consider the task of finding a used car. The data looks like this:

If you’re like me, you filter on mileage, and then sort on price, lowest first. Like so:

You’d probably filter on other things too, but I’m going to keep it simple for now.

Given the options available to us, this is a good place to start, but it also sucks. I’ll let this picture do the talking:

So, inevitably, to find what I’m looking for, I scan through pages and pages of results, go back and twiddle filters, and repeat.

When you add in more dimensions like model year and distance, the problem becomes intractable.

I thought databases were supposed to do the searching. Why am I doing all the work? What I’d really prefer to do is set a target price and mileage, and let the system find the good stuff for me. Like this!:

This mimics the way I actually want to search: first, give me results that meet my targets and are good deals, and gradually transition to good deals that diverge from my targets and into the worse deals that still match them.

Why don’t our search systems do this for us? Well, for one reason, it’s technically expensive. Many relational and NOSQL databases allow you to order by custom functions, and we can express the search above as a scoring function (think along the lines of “order by price * mileage”). But, typically, the database has to calculate the score for every possible result, so it can be slow. My latest open source project is a database designed to make this use case more efficient. If you are development-minded, I encourage you to check it out here: https://github.com/pschanely/scoredb. At scale, scoredb can outperform traditional methods by orders of magnitude.

But I think there is another, more important, reason these search interfaces don’t exist. People just don’t get them.

I’ve built a prototype that does the optimized used car search I described above. A beta version is at http://autofoo.com. Its search interface looks like this:

What could be simpler? I showed this to a few people. I waited for the revelation to wash over them. It did not happen. Some did not understand what was happening. Others didn’t see how this search would produce better results than any other product.

Objectively, I believe, this is the simplest, most intuitive, and most efficient interface. But somehow, databases have trained us to think exclusively in terms of one dimensional sorts and filters.

This is not just about cars; it’s any multi-dimensional search where the dimensions are in competition. It’s about Yelp (rating vs distance). It’s about Amazon (price vs rating). It’s about finding your next apartment (price vs commute vs size and more). It’s about finding your next date online (oh the tradeoffs!).

Let’s stop letting our databases tell us how to find things. Our time is more valuable than theirs.