Monitor your memory usage in real time using AWS IoT

Published in

xTECH

8 min readJan 29, 2018

Ten days ago I entered into the realm of Amazon Web Services and boy was it overwhelming! I began researching about the AWS IoT service by reading their official documentation and unfortunately it wasn’t the best “dummies” friendly documentation. I also tried to look for blog posts regarding the matter, but there wasn’t very much content available. That’s when I made a commitment to write a blog post once things get clearer.

When I became a bit familiar with the very basics, my CTO suggested that I experiment prior to our upcoming project. We then decided to simulate a device that sends sensor readings in real-time and view the messages in AWS IoT MQTT client. We further agreed to monitor a laptop’s RAM usage in real-time using AWS IoT. So here I am sharing with you my experience.

Together we’ll go through this tutorial in baby steps and you should be done with it in less than half an hour.

MQTT (Message Queuing Telemetry Transport)

MQTT is a publish-subscribe-based messaging protocol. This protocol is largely adopted in the IoT world because of it’s ease of use, short response time and relatively lower battery and bandwidth usage in comparison to HTTP.

You can divide MQTT into 3 main components:

Message Broker
Clients
Topics

To explain how these work together, let’s use the chat app analogy.

The Broker here is the chat app, it’t responsible for authentication and creating the whole chat environment. The Clients are the different users of the chat app. Finally, Topics are chatting channels or groups where different users agree to “tune” into in order to communicate.

The “tuning into” a channel is the core of MQTT. Users are anonymous to each other as they can’t add nor have a record of each other like in a conventional chat app. But, they can Subscribe, Publish or do both to a channel in order to communicate. Which means the connection is only between the client and the broker.

In order for you to send a message to your friend you’ll publish your message on a specific topic, and in order for your friend to receive it he will need to be subscribed to that topic beforehand. You will not be able to receive anything from your friend unless you are already subscribed to the topic he/she is publishing on. Also, many users can subscribe to a single topic and receive whatever is being published on it and vice versa.

However, you might ask yourself: how do we control who subscribes to who? What if a certain client is not supposed to publish on a certain topic?

That is where Policies come in, but more on that later.

AWS IoT Features

For you to connect a device — or a simulation of a device— to the AWS IoT, you need three important elements:

A Thing

A Thing is the logical representation of a device on the AWS IOT cloud. In your case it will represent a basic device simulator, the RAM usage monitor, that you will build using the AWS IoT Node.js SDK for Javascript.

A Certificate

Certificates allow the broker to authenticate your device so that the IoT service can identify it and block other devices from connecting to your IoT network. With each certificate AWS IoT will provide you a public key, a private key, and an X.509 SSL certificate.

A policy

If certificates are used for authentication, then Policies are used for authorization. Policies are attached to certificates, so when a device is authenticated, the broker will be able to define which operations or resources that device can access. Basically, it will make sure your device doesn’t do something it isn’t supposed to do.

Memory Usage

You will use the os.freemem()and os.totalmem() methods provided by the Node.js os module to monitor our RAM usage in bytes. They return the amount of free system memory and the total amount of system memory respectively. Using IoT’s Node.js SDK for Javascript you will be able to publish the readings to an MQTT topic in the cloud in intervals of 1 second. Using the MQTT Client UI in your AWS IoT console you will subscribe to that topic and monitor your memory usage in real-time.

Down to business

I’ll assume that you already have an Amazon Web Services account. If you don’t, you can signup for free but keep in mind that you’ll be asked for a valid credit card in order to complete your registration.

Without further ado, let’s begin.

Part 1: Cloud implementation

In the menu on the left click on “Manage” >“Things” > “Create”.
You only need one thing, so click on “Create a single thing”.
Give your thing a name “ramUsage-reader”. Type, Group and Attributes are not important in this tutorial. You can leave them as they are.
On the next window, click “Create certificate”.
AWS IoT will create 4 files for you to download, a certificate for this thing, a public key, a private key and a root CA for AWS IoT. Make sure you download them all before leaving that page, then click “Activate” > “Done”.
In the main page click on “Secure”>“Policies”> “Create”.
Name your policy, then in the “Action” field type “iot:*” to give the policy holder access to all actions such as subscribe, publish and more.
Each “action” has a corresponding “Resource”. Since you selected all actions above, select all resources by inserting “*” in the “Resource ARN” field.
Check “Allow” under “Effect” then click “Create”.

Up to this point you created a thing, a certificate and a policy. To bring them all together you will attach both the thing and the policy to the certificate.

Go back to the main page and click “Secure”>“Certificates”.
On the certificate card that you just created, click on the three dots. On the dropdown menu that just appeared click “Attach thing” then select your thing “ramUsage-reader” then click “Attach”.
Again, click on the same three dots select “Attach policy” and select the policy you just created.

Now that our cloud is all set up, let’s prepare our device simulator.

Part 2: Device setup

Download Node.js pre-built installer for your platform and install it.
If you already have Node.js installed, make sure it is version 4 or above by running node -v in your terminal.
Create a new directory an cd into it:

mkdir ramUsage-monitor
cd ramUsage-monitor

Create a package.json file:

npm init -yes

The -yes is added to use defaults and not being prompted for options.

Install AWS IoT Node.js device SDK:

npm install --save aws-iot-device-sdk

Create a sub directory, name it “certificates” and put in it the 4 files you downloaded earlier. If you’re planning to push your work on github or any similar service, make sure you add /certificates to your .gitignorefile.

mkdir certificates

Create a new file index.js.

touch index.js

Now our ram-usage-monitor directory should look like this:

/
  |- certificates/
    |- xxxxxxxxxx-certificate.pem.crt
    |- xxxxxxxxxx-private.pem.key
    |- xxxxxxxxxx-public.pem.key
    |- Ver...Authority-G5.pem
  |- node_modules/
  |- index.js
  |- package.json
  |- package-lock.json

In your index.js import the IoT SDK and the Node.js os module.

const awsIot = require('aws-iot-device-sdk');const os = require('os');

Setup your device as following:

const device = awsIot.device({
  keyPath: "./certificates/xxxxxxxxxx-private.pem.key",
  certPath: "./certificates/xxxxxxxxxx-certificate.pem.crt",
  caPath: "./certificates/VeriSign........Authority-G5.pem",
  clientId: "ramUsage-reader",
  host: "<YOUR THING ENDPOINT>",
  region: "<YOUR REGION>"
});

To get the API Endpoint of your thing click “Manage”> “Things”, then select your thing and go to “Interact”. It looks like this:

xxxxxxxxxxxxxx.iot.<REGION>.amazonaws.com

Add the following helper class to start/stop a timer that we will use to send the readings in intervals:

function Timer(fn, t) {  var timerObj;  this.stop = function() {
    if (timerObj) {
      clearInterval(timerObj);
      timerObj = null;
    }
    return this;
  }  this.start = function() {
    if (!timerObj) {
      this.stop();
      timerObj = setInterval(fn, t);
    }
    return this;
  }}

Create a timer from the class above:

let sendUsage = new Timer (()=>{
  device.publish('ram', JSON.stringify({
    totalMemory: os.totalmem(),
    usedMemory: os.freemem(),
    timeStamp: new Date()
  })
)},1000)

Calling .start() on the sendUsage object will make our device publish to the topic ram a message composed of our total memory, used memory and the date and time they were measured at every 1 second.

Finally, pass the connect event a callback that will log ‘connect’ and start the sendUsage timer:

device
  .on('connect', function() {
    console.log('connect');
    sendUsage.start()
});

Your index.js file should look like this now :

const awsIot = require('aws-iot-device-sdk');const os = require('os')const device = awsIot.device({
  keyPath: "./certificates/xxxxxxxxxx-private.pem.key",
  certPath: "./certificates/xxxxxxxxxx-certificate.pem.crt",
  caPath: "./certificates/VeriSign........Authority-G5.pem",
  clientId: "ramUsage-reader",
  host: <YOUR THING ENDPOINT>,
  region: <YOUR REGION>
});function Timer(fn, t) {  var timerObj;  this.stop = function() {
    if (timerObj) {
      clearInterval(timerObj);
      timerObj = null;
    }
    return this;
  }  this.start = function() {
    if (!timerObj) {
      this.stop();
      timerObj = setInterval(fn, t);
    }
    return this;
  }}let sendUsage = new Timer (()=>{
  device.publish('ram', JSON.stringify({
    totalMemory: os.totalmem(),
    usedMemory: os.freemem(),
    timeStamp: new Date()
  })
)},1000)device
  .on('connect', function() {
    console.log('connect');
    sendUsage.start()
});

Part 3: Results

At this point, you have enough code to see some results. Head back to your AWS IoT console and click “Test” on the left side. Here you can publish and subscribe to topics to test your implementation.

Now that you have set your device to publish a message to the topic ram every second, if you subscribe to the same topic you will be able to monitor your memory usage readings in real-time.

In the Subscription Topic field, type in “ram”, then click on “Subscribe to topic”.
In your ram-usage-monitor directory Run your index.js file:

node index.js

Now you can see your RAM usage in real-time:

And just like that you’ve built your very own AWS IoT application.

Future work

In a future post we will improve what we have already built by:

Controlling when to start and stop receiving readings through publishing different messages on a dedicated topic.
Initially seting up our device to talk to the broker over WebSockets.
Storing the readings on MongoDB using AWS Lambda functions.

If you found this tutorial useful, please click the Clap button, and follow me for more.

Thank you for reading! Leave a comment below if you have any feedback.

Until next time, happy coding!