Don’t Lose Track of Important AWS Cloudtrail Logs from your Kibana Dashboard
Why do we need to use Kibana? A few years have passed, and I am now sick of Kibana. On top of Elasticsearch, Kibana provides an easy way to create meaningful visualizations on top of complex data.
Microservices at OY! Indonesia produce millions of logs a day, and Kibana has helped us to derive insights quickly on what has been going on in our microservices and why.
Thanks Kibana, we are not gonna do your ancient way of “grep-ing” each server log anymore :)
Since we are hosting our services at AWS and we use Cloudtrail in AWS which is a service that enables us to continuously monitor, and retain account activity related to actions across our AWS infrastructure. Analyzing thousand of data every day is obviously a real challenge in AWS Cloudtrail dashboard.
Aggregating these logs into our existing ELK (Elasticsearch Fluentd Kibana) solution allows us to tackle this challenge by giving us the ability to analyze and visualize the data in an easy way.
Objectives
- Aggregate logs into a single location
- Parse and enrich data
- Visualize important event activity
- Get warned for an alert and react faster
- [BONUS] — PCI DSS / ISO 27001 security compliant
Architecture
Dashboard Outlook
I am pretty sure we could agree that Kibana provides better data visualisation then what Cloudtrail does.
Through the automated pipeline stream that we define above (revisit the “architecture” part), every single event line written to the Cloudtrail will appear immediately as well as to our existing logs monitoring platform, i.e. Kibana.
Being aware of any important AWS Cloudtrail security event
Say that we want to get early warning if someone tries to modify one of our production Security Group, revoke our access and authorize another security group rules.
This is where Elastalert comes in to play. We can write alerting rules that match our specific use cases and when the rule is matched we can notify people through slack, yes SLACK!
Other alert integrations which are supported by Elastalert are email, JIRA, Telegram, Line, Google Chat, etc. Find the complete list of alert integration for Elastalert here.
Sample Notification that Elastalert send to our slack channel:
We can add a Kibana dashboard link into the slack alerting message for enabling users explore the wider range of timestamp and find another insights on the Kibana.
This Slack notification message also can be expanded to see a complete event log.
With that, I hope you can get insight of what we are doing at OY! We’re creating other rules that will notify us ASAP in case of any application errors.
Ok done, I just want to make this story light (not require more than 5 minutes reading) :)
We will publish the detailed implementation in a separate post. Stay tuned! :)
Cheers!
