REST API with node.js and elasticsearch

Today, web application development is a highly complex task, where it is crucial to meet the growing number of requirements without sacrificing performance. That is why, in order to stand out in this very competitive field, it is necessary to include more tools to our technological stack.

One of those tools is elasticsearch, which in my last weeks of work has been very useful. But, before going deeper into it, what is elasticsearch? According to its official site “Elasticsearch is a distributed and open source analytic and analysis engine for all types of data, including textual, numeric, geospatial, structured and unstructured”. There, documents are stored in JSON format, and saved in groups called indexes, which are the equivalent of MongoDB collections or relational database tables. Among their main functions are

  • Application search
  • Web site search
  • Business Search
  • Logging and log analysis
  • Metrics and container monitoring
  • Business Analytics

Elasticsearch has official clients in several languages, including: Java, Javascript, PHP, Python, .NET. In this article we will use the Javascript client, so we will need to have Node.js installed ( https://nodejs.org/en/ ), and we will focus mainly on its website search and logging capabilities.

Elasticsearch step by step

We will also use kibana, which is “a data visualization and management tool for Elasticsearch that provides real-time histograms, pie charts and maps”. We can install it from here. Note that the code shown in this article is publicly available on github at the following url.

2- We will need to create a directory for our project, and there open a console where we will execute the following commands:

3- With the previous commands we install express, which is a library that allows the creation of http servers quickly, and the elasticsearch client for javascript. In addition, we add the nodemon development dependency, which restarts the server when it detects changes in the code.

4- Later we will generate a file index.js in the root of the directory, which will contain the following code shown below, where an instance of the express library is created, and the paths and port to be used are defined.

5- With the previous code we have already created our web server. Now it’s time to create the API. To do this, we will create a file called routes.js, also located in the root of our directory. Here we will first instantiate the express router object and the elasticsearch client that will connect to our local server:

6- Since in this article we want to illustrate the search power in an elasticsearch website, in the same routes.js file we will define some hypothetical products that could exist in an ecommerce (the usefulness of this will be seen later).

Another key feature of elasticsearch is logging and log analysis, which can help our development team to have a better traceability of the use of an application and its API, and also a better control over the errors that can occur. It is with that purpose that these lines of code are added:

What was done in the code above was to “intercept” each http request on our server to add the parameters of that call to the index we call logs in elasticsearch. This will allow us to do some analysis in kibana about our application, and in case there is an error, we will be able to see what data it was produced with in order to make a fix in our code. After this, we will define the CRUD endpoints of our REST API.

7- At this point we will expose the POST endpoint that creates a product, which receives a body with the data to save. The function of the elasticsearch client that takes care of this is index, and it receives as minimum parameters the index (equivalent to collection or table) and the body of the data to be saved.

8- Later we can see the GET endpoint that returns a product with a particular id, the one specified in req.params.id. It should be noted that the id to be sent is the one given by elasticsearch. In this case the function of elasticsearch used is get, and it receives the index and the id of the document that is searched.

9- Then we have the endpoint PUT, which updates a specific document based on the id it has in elasticsearch, this function receives the index, the id, and the content to be updated.

10- The penultimate place we have the endpoint DELETE, which receives the id of the document to be deleted.

11- And we have saved the best for last, the GET endpoint without a particular id. Why do we say it’s the best? Because here we can see in action the power and speed of elasticsearch. This endpoint has the option to receive a query string found in req.query, which if it exists will perform a text search in elasticsearch in all the fields of the index and return the documents that match the search. In case it doesn’t receive the query string it will simply return all the documents in that index.

To get our server running we need to run the npm run dev command in the directory where our code is hosted.

Do you remember the products defined initially? It’s time to use them. Let’s imagine that we are an end user looking for furniture in an ecommerce. We see that there are filters in our search like price, brand or color. But we are more picky, and we want to filter by the shape or style of our desired furniture. That will not be a problem if the API includes elasticsearch in the search, because, if we do a search with the word “round”, it will return a product that could be the one we are looking for. Or if we want to filter by the material, “leather” in this case, it will do the appropriate search. You can see the results of these queries in the requests made with postman in the following images.

Clearly this article is just a sample button of the potential that elasticsearch can have in your web services, since it can even be used as a secondary database in text searches where the response speed is critical.

In YOM, we are looking for the incorporation of new technologies in our stack that allow us to provide a better service to our customers, with the aim of producing quality results efficiently through artificial intelligence. To learn more about our team and value proposal, please visit our website, where you can contact us to implement these improvements in your business.

YOM.

Inteligencia Artificial al servicio de tu negocio.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store