Can Meilisearch replace TicketSwap’s Elasticsearch powered events and artists search?

The short answer is: No 😢. Well, not yet! In this post, I’ll describe my investigation and reasoning behind this conclusion. Normally this would have been an internal document, but there is no reason not to share this publicly. Search is hard, and at TicketSwap we have lots of experience with this subject from both a UX and technical perspective. Hopefully, you will find this information useful.

About Meilisearch 😍

I love simple things. It’s the reason I fell in love with Meilisearch. They describe themselves as “An open source, blazingly fast and hyper relevant search-engine that will improve your search experience.” Very promising!

Current alternatives — Elasticsearch and Algolia — have complex setups and are not affordable. The people behind Meilisearch understand this well. In my opinion, they fill a gap with huge potential. Very well done.

TicketSwap’s current Elasticsearch 476-line configuration 🤯

Search is one of the core features of TicketSwap. In 2019, we had over 10M searches. It contains a couple of concepts: events, artists, venues, and cities. You can try it out on our homepage and you’ll see it works pretty well:

Next to basic search terms, the following phrases are supported:

• ‘harry styles amsterdam’
• ‘harry styles march’
• ‘herry styles’

There is also room for improvement though. Some examples:

• ‘harry styles concert’ has no results
• ‘mystery land’ has no results, while ‘mysteryland’ does
• ‘possesion’ has no results, while ‘possession’ does

This all looks pretty straightforward and is what you expect from TicketSwap’s search. Getting there was not that straightforward. We use Elasticsearch and our config became far too complex. It consists of 326 lines of config, and 150 lines to describe the query. The original version was built 4 years ago, and looking at the current configuration I’m lost. The way it works — with analyzers and filters upon indexing — make things especially hard. To give you an impression, here’s a part of our analysis property:

It’s hard for us to make improvements, without getting deep into the Elasticsearch documentation again.

Meilisearch experiment with just a couple of lines of configuration 🙌

Before playing around, I came up with these potential benefits to move away from Elasticsearch and use Meilisearch instead:

• Looking at the examples, the actual search results look promising.
• It is blazing fast.
• An easy configuration that just works.
• Makes our setup accessible to every (backend) engineer.

Let’s start hacking and testing!

Pitfall 1: maximum payload size

The best way to test Meilisearch is by importing all production data. With the maximum payload size, the files had to be split using json-split (finding 10000 was an ideal size).

npm install json-split
cd ./data/exports
for file in ../*
do
  json-split --json=$file --size=10000
done
for file in *
do
  curl \\
    -X POST '<http://127.0.0.1:7700/indexes/public/documents>' \\
    --data @$file
done

I could have used MEILI_HTTP_PAYLOAD_SIZE_LIMIT, but somehow I did’t get this working properly.

Pitfall 2: don’t use multiple indexes if you want a unified search

Or read the documentation first 🙈 And of course I didn’t do that! Diving in, full curiosity and excitement, I created multiple indexes to store our data properly: events, artists, cities, and locations, all with their own properties. After managing to import production data, it was time to run some first searches. It turns out that it's only possible to search within one index. That won't work for TicketSwap, as we want to have a unified search.

My unified search strategy

In order to build a unified search approach, I started trying different strategies. Eventually, I came up with the following strategy. The best way to describe this is by showing example data for all four types of documents. All have been imported into one index called public.

{
  "type": "artist",
  "type_order": 0,
  "id": "bde2b1a3-2532-4a59-92e5-15e553cceb43",
  "country": null,
  "title": "Adam Beyer",
  "popularity_score": 2358
},
{
  "type": "city",
  "type_order": 1,
  "id": 3,
  "country": "NL",
  "title": "Amsterdam",
  "popularity_score": 45106
},
{
  "type": "location",
  "type_order": 2,
  "id": 2,
  "country": "DE",
  "title": "Berghain",
  "keywords": "Berlin",
  "popularity_score": 0
},
{
  "type": "event",
  "type_order": 3,
  "id": "2e502d38-2820-47ed-89e5-ba18920932e0",
  "country": "NL",
  "title": "Awakenings Festival 2021 | 20th Anniversary",
  "keywords": "Recreatiegebied Spaarnwoude, Spaarnwoude, festivals",
  "event_start": "2021-06-26 12:00:00",
  "popularity_score": 2191
},

Meilisearch configuration

I came up with this configuration:

• As you can see, there are two searchable attributes: title and keywords. The idea behind this is to make title leading, but also being able to search documents through less important keywords. The best example here is events. By adding the location name and city to the keywords attribute, searches like 'harry styles amsterdam' work.
• type_order: This attribute makes sure we can give different weights to the different types of documents: artists (0), cities (1), locations (2), and events (3). The trick here is to add asc(type_order) to the ranking rules. This makes sure that searches like 'harry styles' show the artist page above events, and 'amsterdam' the city page above events.
• popularity_score: This is just an internal score calculated by our data team, based on lots of factors like supply and demand.
• Other ranking rules: I came up with this order by reading the documentation and trying out what feels right in our case.

It works amazingly

The configuration looks straightforward and understandable to any engineer. Here are some examples that show it works really well:

Things to improve

Based on this initial investigation — and with the help of our awesome customer experience team — I wrote down a list of improvements I want to make:

• Support special characters like ‘&’ and ‘+’ by using the synonyms setting
• Support date searches (‘harry styles 27 march’) by adding the date to the keywords attribute.
• Support country searches (‘harry styles netherlands’) by adding the country to the keywords attribute.
• Make id a distinct attribute by prefixing the ID with the type (i.e. 'location-123' and 'city-123').

All these things are easy to solve with the simple configuration Meilisearch provides.

The missing killer feature for TicketSwap: Geo Search 🌍

The search results are amazing. The configuration is straightforward. It is blazing fast! The answer to the initial question — Can Meilisearch replace TicketSwap’s Elasticsearch powered events and artists search? — must be ‘yes’ right?

Unfortunately, there is one major feature missing: the ability to rank results based on the location of the user. For a worldwide platform like TicketSwap, this is crucial. Imagine you live in Australia and are searching for ‘harry styles’. You don’t want Harry Styles at Ziggo Dome (Amsterdam) in the top results, but Harry Styles at Rod Laver Arena (Melbourne). That’s currently not possible with Meilisearch. Luckily it’s on the roadmap.

For now, we need to wait for ‘Allow Geo Search’ to arrive 🙏 and give a big shout out to the people behind Meilisearch 👏

If you have feedback about our search engine as a TicketSwap user, or feedback about this post, please send me a DM.