AWS Monitoring Toolkit: Grafana CloudWatch Datasource Setup with IAM Roles

Struggling to monitor your AWS resources effectively? This guide will equip you with the skills to set up a CloudWatch datasource in Grafana and gain actionable insights in no time!

To integrate CloudWatch with Grafana, you’ll need to choose an appropriate authentication method. This choice depends on your Grafana environment and security considerations. In this article, we’ll focus on using Workspace IAM roles, which offer enhanced security benefits compared to other options.

Key Security Advantages of Workspace IAM Roles:

• Using IAM roles Eliminates the risks associated with storing and managing access keys or secrets.
• Grafana automatically acquires short-lived credentials, minimizing exposure in case of breaches.
• Assign specific IAM permissions to the role, ensuring tight control over CloudWatch access.

Let’s configure our CloudWatch datasource in Grafana, with these simple steps.

AWS IAM Role Setup

Step 1

Create an AWS IAM role and attach the policy below, this policy allows the IAM role to fetch logs and metrics from the CloudWatch service, read ec2 instance tags, And allow access to reading tags of other resources. You can read more about this here

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "*"
        },
        {
            "Sid": "AllowReadingMetricsFromCloudWatch",
            "Effect": "Allow",
            "Action": [
                "cloudwatch:DescribeAlarmsForMetric",
                "cloudwatch:DescribeAlarmHistory",
                "cloudwatch:DescribeAlarms",
                "cloudwatch:ListMetrics",
                "cloudwatch:GetMetricData",
                "cloudwatch:GetInsightRuleReport"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AllowReadingLogsFromCloudWatch",
            "Effect": "Allow",
            "Action": [
                "logs:DescribeLogGroups",
                "logs:GetLogGroupFields",
                "logs:StartQuery",
                "logs:StopQuery",
                "logs:GetQueryResults",
                "logs:GetLogEvents"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AllowReadingTagsInstancesRegionsFromEC2",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeTags",
                "ec2:DescribeInstances",
                "ec2:DescribeRegions"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AllowReadingResourcesForTags",
            "Effect": "Allow",
            "Action": "tag:GetResources",
            "Resource": "*"
        }
    ]
}

Step 2

Add your ec2-instance ARN to the Grafana_CloudWatch role trust relationship. This will allow your instance to be able to assume the IAM role you just created using sts:AssumeRole.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::025xxxxxxxxx8978:role/your-instance-arn-202306146573856549600000005",
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Grafana Datasource Setup

With your IAM Role created let's set up CloudWatch as a data connection in Grafana, Instead of using the user interface, Grafana allows you to define and configure data sources directly within YAML files. We can add our CloudWatch data source connection using either option.

apiVersion: 1
datasources:
  - name: CloudWatch
    type: cloudwatch
    jsonData:
      authType: default
      assumeRoleArn: arn:aws:iam::123456789012:grafana-cloudwatch
      defaultRegion: eu-west-2

Or you can head over to the Grafana UI and follow the steps below

— Hit the hamburger menu, tap on connections and select “add new connection”

— Search for CloudWatch and select “add new datasource”

— Use AWS SDK Default as the authentication provider, input your newly created IAM role ARN and, your Default Region.

— Scroll down to test your connection you will get the below success message if your setup was successful.

And with that, your CloudWatch Data connection is set up! you can now create awesome Dashboards using metrics pulled from CloudWatch, Check Awesome CloudWatch Dashboards you can replicate on Grafana.

References

Grafana Documentation

Grafana CloudWatch dashboards