Get Your Amazon Cloud Watch Logs Today!

Best way to Manage your Logs using WSO2 Integration Cloud

This is my second week of Internship at WSo2 and I have implemented a simple dashboard which can retrieve and show the logs from AWS CloudWatchLog.

Dashboard to show Log Groups, Log Streams and Log Events with Filters

AWS CloudWatchLogs API

I have used AWS CloudWatchLogs API with Node.js and Angular.js to develop this dashboard. Amazon docs give better documentation to implement a Node.js application with AWS SDK. You can follow this documentation to get more information about AWS.CloudWatchLogs API.

Development Coverage

I have developed back-end with Node.js and It can retrieve the data from AWS CloudWatchlogs using REST APIs. There are several APIs which can play a major role in this development.

  1. Cloudwatchlogs.describeLogGroups() — Used to retrieve Log groups from AWS CloudWatchLogs.
    Log groups — groups of log streams that share the same retention, monitoring, and access control settings. Each log stream has to belong to one log group (source).
    Limits : You will get only 50 log groups per a request with nextToken. So you can get next set of log groups using that nextToken.
  2. cloudwatchlogs.describeLogStreams() — Used to retrieve Log Streams under given Log Group from AWS CloudWatchLogs.
    Log streams — A log stream is a sequence of log events that share the same source. More specifically, a log stream is generally intended to represent the sequence of events coming from the application instance or resource being monitored (source).
    Limits : You will get only 50 log streams per a request with nextToken. So you can get next set of log streams using that nextToken.
  3. cloudwatchlogs.filterLogEvents() — Used to retrieve Log Events under given Log Group and set of Log Streams (max 100) from AWS CloudWatchLogs.
    Log Events — A log event is a record of some activity recorded by the application or resource being monitored. The log event record that CloudWatch Logs understands contains two properties: the timestamp of when the event occurred, and the raw event message (source).
    Limits : You will get only 10000 log events per a request with nextToken. So you can get next set of log events using that nextToken. You can give only 100 log streams per request as an array.

I have implemented all the REST calls for the necessary actions using Node.js(Express framework) and Stared the server.

Now I can get the Logs from AWS CloudWatchlogs to my client PC — As JSON Objects …!
JSON Object for the Log Events request

Now, I wanted to display those JSON Objects to the User, So I advised to implement using Jaggery, But for the demonstration part, I have started to build using Angular — because I already familiarized with Angular.

Dashboard Log Groups page

WSO2 Integration Cloud

One of the cloud service under WSO2 is Integration Cloud. WSo2 just implemented their Integration product in Cloud to make it easy for their customers.

You can just go to to access WSO2 Cloud and you will be given 14 days trial period at the beginning. But unfortunately, If you have a trial license, then your deployed application will be turned off after 8 hours from that application’s deployed time.

You can just Sign up using your Email, No more requirements :-)

After the successful login/signup, you will be redirected to WSO2 Cloud Dashboard.

Cloud Dashboard of WSO2 Cloud

Just Click on Go under WSO2 Integration Cloud and then, you will be automatically redirected to WSO2 Integration Cloud dashboard. WSO2 provides great UI/UX for their customers with reduced complexity. You can see, there are only a few functionalities(buttons) are available for the customers!

Just Click on Create New Application. Then you can see there is some support applications list.

Currently, they are providing Integration Support for,

  1. WSO2 ESP Composite Application — Allows you to deploy an ESB configuration that is supported by WSO2 Enterprise Service Bus
  2. Ballerina — Allows you to deploy an integration scenario developed by Ballerina.

And they are providing Service support for,

  1. Java Web Application — Allows you to create dynamic websites using Servlets and JSPs and deploy web services.
  2. Java Micro Service — Allows you to create dynamic websites using Servlets and JSPs and deploy web services.
  3. PHP Web Application —Allows you to create dynamic websites using Servlets and JSPs and deploy web services.
  4. WSO2 Jaggery Application — Allows you to write all parts of web applications, services, and APIs in a completely JavaScript way.
  5. WSO2 Data Service —Allows you to deploy a data service that is supported in WSO2 Data Services Server.
  6. Node.js Application —A llows you to deploy a data service that is supported in WSO2 Data Services Server.

Deploy Amazon CloudWatchLogs Application into Integration Cloud

Just create Node.js Application to deploy this Amazon CloudWatchLogs Application.

One Minute! You want to include your Node.js route file/initial file as Index.js. Because when Integration cloud tries to start your application, It will try to start from Index.js.

When you moved to create New Node.js application, you will be required to Upload Zipped file or provide URL for the application.

Are You new to WSO2 Integration Cloud, then just click on Try out Samples ….

Then after, you will be required to provide relevant information such as Application Name, Version and Description for your deployment. Finally, WSO2 Integration cloud will create Docker images for your application and start to build your application in Cloud.

What is Docker? Just follow my blog post about the Dockers !

After the creating the application, You will be redirected to the application Dashboard which will give more information about your deployment. There will be an auto generated URL for your Node Application and you can just click on that to start your Application.

Cool… Now Amazon CloudWatchLogs Application is running on WSO2 Integration Cloud …!