AWS CloudWatch Alarms on Microsoft Teams

sebastian phelps
5 min readFeb 1, 2019

--

AWS offers a pretty neat way of being warned about things going wrong in your infrastructure, but if you’ve used it then you may be aware that the emails it sends out can be a little abstract and email might not always be the right place for your team to see and discuss an issue.

I was looking for a way to get human readable messages to Microsoft Teams, which happens to be what the company I work for most regularly use. Integration with slack seems pretty forthcoming, but I couldn’t find much around this topic for Teams.

So let’s get started. We are going to:

  • Make sure the cloudwatch alarms are sending messages to SNS (if you are already using AWS CloudWatch alarms for emails, they already are)
  • Find out the ARN for the alarm SNS topic
  • Configure a lambda which is triggered by the SNS topic
  • Add the ‘Incoming Webhook’ connector to our Teams channel
  • Add the webhook url as an Environment variable to the lambda
  • Test out the lambda
  • Relax

On the AWS side, against an alarm you can add an Action. Creating a new list actually creates a new SNS topic. Messages will be written to the topic when the state changes to the one you specified. Assuming you want to know when something goes wrong, and when its back right again you should create an action for both ALARM and OK and have it send a message to your topic. e.g. production-issues.

Head on over to the SNS service, and you should have your topic there, make a note of the ARN.

Now we need to create a lambda, which will be fired every time a message is posted to that topic, and who’s job itis to create a nice readable message and post it to Microsoft Teams.

So head on over to the Lambda service, and create a new function, choosing ‘Author from scratch’ and selecting ‘Python 3.6’ as the runtime.

When adding the role, the default permissions you are offered which allow your function to log, should be enough.

Add the trigger ‘SNS’ and add the ARN we got earlier as the topic.

Now to configure the actual function, click on the lambda function in the designer panel, and add the following code.

import json
import logging
import os

from urllib.request import Request, urlopen
from urllib.error import URLError, HTTPError

HOOK_URL = os.environ['HookUrl']

logger = logging.getLogger()
logger.setLevel(logging.INFO)


def lambda_handler(event, context):
logger.info("Event: " + str(event))
message = json.loads(event['Records'][0]['Sns']['Message'])
logger.info("Message: " + str(message))

alarm_name = message['AlarmName']
old_state = message['OldStateValue']
new_state = message['NewStateValue']
reason = message['NewStateReason']

base_data = {
"colour": "64a837",
"title": "**%s** is resolved" % alarm_name,
"text": "**%s** has changed from %s to %s - %s" % (alarm_name, old_state, new_state, reason)
}
if new_state.lower() == 'alarm':
base_data = {
"colour": "d63333",
"title": "Red Alert - There is an issue %s" % alarm_name,
"text": "**%s** has changed from %s to %s - %s" % (alarm_name, old_state, new_state, reason)
}

messages = {
('ALARM', 'my-alarm-name'): {
"colour": "d63333",
"title": "Red Alert - A bad thing happened.",
"text": "These are the specific details of the bad thing."
},
('OK', 'my-alarm-name'): {
"colour": "64a837",
"title": "The bad thing stopped happening",
"text": "These are the specific details of how we know the bad thing stopped happening"
}
}
data = messages.get((new_state, alarm_name), base_data)

message = {
"@context": "https://schema.org/extensions",
"@type": "MessageCard",
"themeColor": data["colour"],
"title": data["title"],
"text": data["text"]
}

req = Request(HOOK_URL, json.dumps(message).encode('utf-8'))
try:
response = urlopen(req)
response.read()
logger.info("Message posted")
except HTTPError as e:
logger.error("Request failed: %d %s", e.code, e.reason)
except URLError as e:
logger.error("Server connection failed: %s", e.reason)

To get custom readable alarms, you will have edit this a little. The messages variable contains the mapping between alarm name and the message you want. If there isn’t anything in there it will default to some useful information but I can’t promise it will be very readable.

Underneath the function code you will have to define one Environment Variable, HookUrl, this is the url for your connector on teams.

To get your connector url head over to Teams, and click on menu next to the channel you want the message to appear in. This should give you an option to add connectors. We want to use the ‘Incoming webhook’ connector, which basically gives you an endpoint which you can send any message you like to that channel.

Configuring the Incoming Webhook should give you a url which you will need to put in the HookUrl Environment variable.

The lambda function we are creating will use Connector cards to send messages to Teams. You can get more information from the docs.

Once you’ve entered the HookUrl you got from teams, you can save the lambda, and you are ready to test it out.

So lets test the lambda bit atleast, at the top choose to configure test events.

Here is some sample data which will mimic what SNS will send to our lambda.

{
"Records": [{
"Sns": {
"Message": "{\"AlarmName\": \"my-alarm-name\", \"OldStateValue\": \"OK\", \"NewStateValue\": \"ALARM\", \"NewStateReason\": \"just because\"}"
}
}]
}

You can trigger that test data and hopefully… you should see your alarm appear in your Teams channel.

--

--