Why We Love Elasticsearch

Tim Tutt
Tim Tutt
May 22, 2018 · 5 min read

It’s The Perfect Fit

Image for post
Image for post

In a previous post we discussed how data modeling always starts with the questions you want to ask of your data. We tried to keep that post clear of anything that could start a technology war, but in this one we’re going to take a very strong stance.

Image for post
Image for post

The thing is, we have a confession to make. Everyone has their favorite tools and we are no different.

Elasticsearch is our first love and it is a perfect fit for ClearQuery.

Don’t worry — we will spend the rest of this post justifying and defending that position.

Buckle up.

ClearQuery was designed from the beginning with a core goal of answering questions. It does this by providing a more conversational experience to end users trying to interact with their data. We believe that you should not have to learn a new query language just to ask questions. Conversing with your data should be as natural as speaking.

With that said — it becomes very important to the user experience to ensure that answers are provided as quickly as possible. The more instantaneous the better. It feels more natural than writing a query and waiting a while before you can follow up with yet another query. It just works.

In a traditional relational structure, the data required to answer the questions you want you ask of your data could live in separate tables. This means, at query time you could have to wait while the query performs various joins across tables. Sure in most cases this is a few seconds, but those few seconds have an adverse effect on the user experience. It becomes even worse as your user base starts to grow.

Denormalization shifts when and where the work occurs. Rather than at query time, this wait occurs at indexing time (although with Elasticsearch that’s pretty fast too). As you follow up with questions, the machine does not have to keep doing the complicated parts of joining things because the data is available in every record.

In the case of Elasticsearch, this results in low double (sometimes single) digit millisecond response times — it may as well be instant.

The denormalization of data also allows for very flexible schemas which also aids in the modeling of data for new types of questions that arise as you begin to explore your data. As your schemas change over time to be able to ask new questions.

Added a new numeric field? Now you have the ability to ask a large number of metric related questions around that particular data point and how it relates to the rest of your data. New date field? You can now ask questions about how your data changes over time. Geo coordinates? Ask about where important events are occurring.

Image for post
Image for post

While Elasticsearch’s query DSL may not be the easiest thing to write, it is extremely intuitive. The query DSL, like everything else in Elasticsearch is JSON based, which means it is super verbose. The plus side is that almost anyone can read an Elasticsearch query and understand most of what’s going on.

There are some nuances to the DSL, but this structure is key to programmatically piecing together a query when translating from a natural language question into the query that gets run against the index. ClearQuery takes advantage of this structure and the optimal ways to query data so that your knowledge discovery experience is fantastic.

When people hear the names ClearQuery and Elasticsearch, there is a natural inclination to just think about search. And while both solutions have major search components to them, neither of them is just about search. At their core, they are both designed to help you converse with your data and tell stories with it.

Search capabilities allow you to filter data down to the information that is relevant to the questions you want to ask, but it is the other features that allow you to get more in depth when your data is modeled appropriately.

Aggregations allow you to ask questions like “What was the average number of sales in the North Eastern Region last month?” or “How many visitors did our website receive broken down by country?”. These are the questions that help you derive actionable business intelligence that you can leverage.

Percolation gives you the ability to schedule alerts or automate actions as soon as new data hits your system. “Alert us whenever a website visitor tries to access a restricted section.”

“More Like This” queries are contextually aware and allow you to ask questions like “What customers have similar profiles to this one?”.

And those are just the free things that come out of the box. When you take advantage of X-Pack’s Machine Learning and Graph features you can ask “What are my forecasted sales for the next two quarters?” and “Who is the biggest influencer of our brand on Twitter?”

If you are not convinced by now that Elasticsearch was the perfect fit for ClearQuery, then you haven’t been paying attention. Elasticsearch already gives you the ability to ask these questions, ClearQuery just makes it much easier.

“Any sufficiently advanced technology is indistinguishable from magic”

-Arthur C. Clarke

As far as business leaders are concerned, using ClearQuery is like waving a magic wand to activate the Elasticsearch magic that provides answers to their questions.

And that is why we love Elasticsearch.

Interested in learning more about how ClearQuery can help your business get answers to your questions? Email us at info@clearquery.io or visit our website for more.

ClearQuery

ClearQuery turns natural language questions into…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store