Testing Elasticsearch in Rails

This year I began to work on a few projects in Rails. Besides feeling ten years late to the party, I found it was a smooth transition. Smooth, that is, until it came to integrating Elasticsearch.

For those of you unfamiliar with Elasticsearch, it is a search engine of pure wizardry. It indexes documents and processes queries at mythical speed. For integration with Rails, there is even an official gem that handles just about everything. So if the gem’s good, what’s the problem?

Testing. Unit testing to be exact. An avid user of RSpec, I couldn’t find clear guidelines on how I was supposed to test my integration with Elasticsearch anywhere. After toiling over pages and pages of search results, I came up with a solution that worked for me, and I’d like to share it with you in case it helps you too.

Using a test database

I found numerous articles which suggested namespacing indexes by environment. This would allow you to use the same server for both development and testing but it didn’t seem like a clean approach.

Alternatively, you can use an in-memory instance (or “cluster”) of Elasticsearch just for your test suite. In theory this is faster, and seems cleaner too. The test cluster is provided by the elasticsearch-extensions gem:

Require the cluster at the top of your spec_helper and start it before your test suite runs. You’ll want to stop it afterward, too:

A couple of things to note about the above:

• 9250 is used as the port instead of 9200. This is to avoid collision with your development instance of Elasticsearch.
• We only start a cluster if we need to. By specifying elasticsearch: true, only tests that are tagged with elasticsearch will start a cluster. More on tagging tests soon.

Setting up your indexes

At the start of each test run, you will now have an empty Elasticsearch cluster. Before you can publish any documents to it, you will need to create indexes for each model that implements Elasticsearch. To ensure a clean slate for each test, you’ll want to do to this before and after each is run:

Again, note the use of elasticsearch: true to ensure this work is only done for tests that are tagged with elasticsearch. The code above also suppresses some noise emitted by the Elasticsearch gem whenever an index is forcibly created or destroyed.

This approach is a bit of a hammer. You can be more specific about what indexes you create if you want to be. This example names models directly rather than traversing descendants of ActiveRecord::Base.

Writing tests

Now that your indexes are set up, you can begin to test your integration. Say you had a model, Fruit, that implemented Elasticsearch. Here’s how you might test that Fruit is being indexed properly:

Again, some notes on the above:

• As promised, we have tagged the test with elasticsearch: true. This lets RSpec know that it needs to set up an Elasticsearch cluster and indexes for it.
• It takes Elasticsearch a moment to index new documents. To be sure that your newly created model has been indexed before you search for it, you should call refresh_index!

Gotcha: testing after_commit hooks

If, like me, you rely on callbacks to keep your model indexes in sync, the above test will have failed. This is because after_commit hooks are not triggered in tests. This has been fixed in Rails 5 but you will need to use the test_after_commit gem in the meantime:

That’s it — run bundle install and re-run your tests.

If you are using asynchronous callbacks, your setup will be different. I’m not going to discuss it here but this slideshow may offer a solution that works.

Travis

It’s easy to get your tests up and running on Travis CI. In your configuration for Travis, specify that you require Elasticsearch as a service then add a brief sleep to your before_script clause. This makes sure the instance is ready before your tests run. I use a delay of 5 seconds without any trouble, though their documentation recommends a delay of 10.

Final recommendations

While it’s not directly related to testing Elasticsearch, I recommend using Database Cleaner to ensure a clean state during tests. Installation is as simple as specifying a new dependency:

… and adding the hooks:

Last words

While this solution worked for me, it is by no means perfect. If you have any thoughts on how it could be improved, I’d love to hear from you.