ELK Stack ElasticSearch, Kibana and Node.Js : Store Logs on ELK

The inspiration for this article came from a lack of proper resources to help beginners get an understanding of how ELK stack can be utilized to store logs for a web app. I am here to share the knowledge which I have gained after numerous hours of research.

This integration will be divided into the following steps:

• NodeJS Project Setup
• Winston Logger Installation
• Setup ElasticSearch
• Running it

Step 1: Node.JS Project Setup

For this example, I’m using a basic nodeJS application that has express installed for basic routing. The source code for the project set up can be found here.

After cloning the repository to a folder of your choice. This command will install all the node packages.

npm install

To start the server:

npm run start

The project currently has two endpoints setup and will run the app on localhost:9000.

Step 2: Winston Logger Setup

For this tutorial, I will be using Winston for logging purposes. It is a logger which can be used for output to multiple transports. A transport is a storage location for logs. You can read more about transports here.

To install the Winston packages:

npm run install --save winston winston-elasticsearch @elastic/ecs-winston-format

Under the /src folder, let’s create a file called logger.js.

In the logger.js file, we add the following code.

const winston = require('winston');
const ecsFormat = require('@elastic/ecs-winston-format');
const { ElasticsearchTransport } = require('winston-elasticsearch');
const client = {
  clientOpts: {
    node: 'ELASTIC NODE ENDPOINT', //   Elastic Node Endpoint
    auth: {
      username: 'USERNAME',
      password: 'PASSWORD' 
   }
  }
};
const esTransport = new ElasticsearchTransport(client); 
const logger = winston.createLogger({
  level: 'info',
  format: ecsFormat(),
  transports: [
    esTransport
  ]
});
module.exports = logger;

In the code snippet above, the Winston logger is being initialized with the elasticsearch transport.

Step 3: Setup Elasticsearch

Head to this web page to setup Elasticsearch and create a free account. Once the account is ready, let’s create a new deployment.

Naming the deployment

Once the deployment is created, the username and password are given. Store this password for further use.

Click on the setting button

Click on the copy endpoint and paste this endpoint link along with the username and password that was saved recently to the logger.js file that was created in Step 2.

Step 4: Running it

In the src/router.js, add the following code to start logging with Winston.

var express = require("express");
var router = express.Router();
const logger = require('./logger'); // Get the winston logger from logger.js
// /* MICROSERVICE ROUTER */
router.get("/test1", async (req, res, next) => {
  try {
    logger.error("Test 1 Log");
    return res.jsend.fail({ validation:['example'] });
  } catch (err) {
    res.status(400).send(err).end();
  }
});

router.get("/test2", async (req, res, next) => {
  try {
  logger.log({
    // Message to be logged
    message: 'Test 2 Log',
    // Level of the message logging
    level: 'info'
  });
  return res.jsend.fail({ Testing:['Result1'] });
  } catch (err) {
    res.status(400).send(err).end();
  }
});
module.exports = router;

After updating the code, let’s start the server by running:

npm run start

Go to localhost:9000/test1 and localhost:9000/test2 to log messages for each of the endpoints.

Now, let’s go back to elasticsearch settings page and open kibana.

Once Kibana launches, click on observability.

Click on the stream under the Logs section.

The logs that were just generated should be visible in the stream.

Furthermore, Winston can be configured to include more information for the logs such as objects, etc.

Thank you for reading! ❤

I’ll be writing another article on how to store Docker logs on ELK soon. If you liked this article, please feel free to follow my page for more content.