Message Logging made easy with Apigee Edge
In this post, I have tried to cover most common external message logging system with Google’s API Management — Apigee Edge. At the end of the post you can find link to the code repository that you can use.
Okay, so let’s get started.
Apigee Edge offers a seamless syslog integration to external logging systems like Loggly, Splunk, SumoLogic etc. to log your API requests. However, you can also send logs to ELK (Elasticsearch, Logstash and Kibana) stack.
Although sending logs to Loggly is straight-forward by implementing out-of-the-box Message Logging Policy, however it is little bit challenging when integrating with ELK stack (Elasticsearch, Logstash and Kibana), especially sending JSON log messages to the ELK stack.
I have put some efforts in this and tried my best to cover both the ELK stack (Part 1) and Loggly (Part 2) logging in this post.
The sole objective of creating these repositories is to help Apigee developers QuickStart external logging with as less configurations as possible.
Why?
Logs are quite important to track down issues in API runtime environments. Logs can be useful for developers during and after the development phase, especially because they are the ones writing the code and configuring the APIs. Also, it is important for security teams to see if there has been any unauthenticated/unauthorised API access; logging gives the security team all the information they need.
Apigee Edge offers a 10 min or 20 messages trace window (whichever reaches first), so you can trace the live calls to find the issues or capture messages during development to see whether runtime properties are being set properly or not, for e.g. quota counter for a developer app . However that is not sufficient when the API call was made outside of this trace session. You are helpless if you don’t have the logs. Having an Apigee Private Cloud/On-premise subscription, you have full access to the logs, however having Apigee Edge Public Cloud subscription you do not have access to the logs. So if you happen to be a Public Cloud Customer, it is highly recommended to use either your own premise ELK stack or a third-party log management services like Loggly to have full and complete insights in your APIs.
How?
I have created two git repositories (see Part 1 and Part 2), both contain a working demo/api proxy which demonstrates logging to Loggly (Formatted JSON) and ELK Logging.
Part 1
Apigee Edge — External Message Logging to ELK stack
Repo Link — https://github.com/bhatikuldeep/hello-world-elk-logging
Part 2
Apigee Edge — External Message Logging to Loggly
Repo Link — https://github.com/bhatikuldeep/hello-world-loggly-logging
I have tried my best to cover what is essential, however if there’s any support required, feel free to contact me.
