Handling non-alphanumeric fields in AWS CloudWatch Log Insights
QUICK FIX #6
In QUICK FIX posts I narrowly focus on issues that I’ve came across while working or hobby-ing in software development in the wild. The issues could be bugs; nuances of the program or service in question; or simply my own misunderstanding. Or something in-between. The goal is to quickly help people out who come across the same problem.
AWS CloudWatch Log Insights is a really useful tool built-in to AWS CloudWatch which can be used for analysis on any CloudWatch Log Groups.
I recently got a bit confused when my query wasn’t returning any results when I knew that it should. I kept trying different things as I thought there might be something off with the query logic and was searching around when I found the simple solution.
Any log field named in a query that has characters other than the @ sign, the period (.), and alphanumeric characters must be surrounded by backtick (
`) characters. For example, thefoo-barfield name must be enclosed in backtick characters because it includes a non-alphanumeric character.
This makes complete sense, I simply didn’t know it was the case initially. The field that I was using did contain a special character!
Demo
To demonstrate I have set up a test EKS cluster and enabled logging to CloudWatch. I set a public API server endpoint and I want to query the API server audit logs to see where requests have been disallowed.
Here is a good example of the use of the backticks due to the field containing a forward slash character and also a general good example some some of the query language features.
And here is what the query looks like in action:
Takeaway
Log Insights is a powerful tool where in most cases there will be a way to answer any question that you want to ask your logs. As with any tool, you need to get used to how to use it and it’s quirks. Here is another link to some sample queries which can be a really useful jump start.
