How to Collect Custom Metrics from AWS EC2 Instances

Zeeshan Baig
May 11, 2018 · 4 min read

Monitoring is a critical part of any cloud infrastructure and it is important to maintain reliability, availability, and performance of your AWS cloud applications. There are 2 main types of monitoring you can do on AWS EC2 Instances as follows

  • Basic Monitoring for Amazon EC2 instances: Seven pre-selected metrics at five-minute frequency and three status check metrics at one-minute frequency, for no additional charge.
  • Detailed Monitoring for Amazon EC2 instances: All metrics available to Basic Monitoring at one-minute frequency, for an additional charge. Instances with Detailed Monitoring enabled allows data aggregation by Amazon EC2 AMI ID and instance type.

While limited monitoring is enabled by default, it does not cover the memory utilization or disk I/O of the instances, for these use cases you need to enable custom monitoring on EC2 instances.

This post covers how to enable custom detail monitoring and collect memory and disk metrics using AWS CloudWatch agent, later you can build custom CloudWatch dashboards using these metrics.

Note: You can also monitor EC2 instances using older Perl scripts click here for more info

In summary, you need to do the following:

  1. Create CloudWatch Role
  2. Assign CloudWatch Role to EC2 Instance
  3. Install CloudWatch agent on the EC2 Instance
  4. Configure Metrics
  5. Start CloudWatch agent
  6. Create CloudWatch Dashboards

Following are the details of the above steps

Create CloudWatch Role

Click Role in the left panel and click Create Role button

Select AWS Service, EC2, and Next: Permissions button

Search for cloudwatch and check CloudWatchAgentServerPolicy and click Review

Name your role such as EC2CloudWatchAgentRole and click Create role

Assign CloudWatch Role to EC2 Instance

Install CloudWatch Agent

You can download and install CloudWatch agent using the following commands on Linux (for Windows see AWS Docs)

$ cd /tmp $ wget https://s3.amazonaws.com/amazoncloudwatch-agent/linux/amd64/latest/AmazonCloudWatchAgent.zip $ unzip AmazonCloudWatchAgent.zip $ sudo ./install.sh

Configure Metrics

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

The above command would ask series of questions and generate the configuration file at the following location

The config file is usually located at /opt/aws/amazon-cloudwatch-agent/bin/config.json

To learn more about configuration file click here

Start CloudWatch Agent

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

Create CloudWatch Dashboard

Hope you find this post useful, please leave a comment and share your thoughts.

For more info or troubleshooting check AWS docs

About DataNext


Originally published at datanextsolutions.com on May 11, 2018.

Zeeshan Baig

Written by

Cloud, Big Data, DevOps Expert, Musician & CEO of DataNext Solutions, helping people every day with the latest tech. Connect @LinkedIn http://bit.ly/zb-linkedin