Using Cloudwatch to log web errors

Cloudwatch is AWS’ own monitoring service for resources, but it can do a lot more than that. With built-in alarms and its own query language, it can also be used to receive and visualize logs and errors from the front-end.

How to ingest logs on CloudWatch?

Cloudwatch can’t directly store logs through an API as a traditional bug tracking system can. It can only query “log streams” from internal AWS resources, meaning that we’ll need to put our client logs in some other AWS service before we can query it in CW, like a lambda function.

By default all stdout of the lambda functions end up in their own log group, which is great because we can just have a lambda function that receives JSON payloads from the front-end and logs to the console whatever it is receiving to be queried from CW.

Creating a lambda logger and exposing it

Create the lambda function that will act as the log ingesting component of our setup.

Choose the Node.js runtime and all default configs.

Edit the source code to look like this:

export const handler = async(event) => {
   console.log(JSON.parse(event.body))
   return {}
};

Now that we have the lambda function deployed, let’s expose it though an HTTP endpoint using API Gateway.

1. On the lambda function diagram on the top, click “add trigger”
2. Select “API Gateway”
3. Select “create new” if you have no API gateway configured

AWS has now created the API gateway resource for you.

Next up, we need to enable CORS to allow them to be called from your front-end.

To do so, we need to find out what URL the API is exposed to, and then we can test it out. Head to the API gateway service, find the API that lambda has created for you and go to the “Resources” page to enable CORS on the dropdown after selecting the /log endpoint:

In the same menu, deploy the API again so the CORS settings get changed.

All done! Let’s grab the endpoint URL on the “Stages” tab:

The “Invoke URL” on the top is the one you are going to send your logs to.

With your URL in hands, let’s curl a fake log:

curl 'invoke_url_here' -X POST --data-raw '{"source": "testing", "level": "ERROR"}'

If it doesn’t throw any errors, success! The logging endpoint accepts payloads and it’s already reporting to CloudWatch.

PS: CloudWatch takes a few minutes to process new log lines.

Sending logs from the front-end app

No rocket science here, just send anything you want to the log endpoint and it’s then query-able by CloudWatch. I’d have a function like this in the front-end app:

function log(level, message) {
    const url =
      'https://invoke_url_here/default/log';

    const meta = {
      agent: navigator.userAgent,
      language: navigator.language,
      platform: navigator.platform,
      level: level,
      source: 'web-app',
    };

    fetch(url, {
      method: 'POST',
      body: JSON.stringify({ meta, message }),
    });
}

// ... whenever there's an error
log('ERROR', new Error('Error logging in!').message);

Querying logs on a table on CloudWatch

Now that we have a couple errors ingested, let’s query them in a table on CloudWatch using its query language.

On the console, head to the “Log insights” tab and select the “/aws/lambda/log” log group. Then click on “Run query”.

You can see that there are more log lines than what we need to see, such as memory and time consumed by the lambda logger. To fix that we can filter by only errors from the web app using the following query:

fields @timestamp, message
| filter meta.source like /web-app/
| filter meta.level like /ERROR/
| sort @timestamp desc
| limit 100

Heads up: Fields that start with @ are the internal default fields, so @message refers to the whole payload, while message is the field we added on the payload to contain the error string.

That’s all! You can now query your logs into a graph and create alarms in case new errors come from your front-end.