Open Sourcing TCP plugin for Kong

In last past I talked about how we use capabilities of our API gateway (Kong) to power a real time API dashboard which gives detail insight into performance of our APIs. We get to know API error rates, latencies, requests per seconds for each API. This dashboard is often a starting point for engineers to debug and trace production issues.

As mentioned in previous post, to power this information we use kong’s TCP logger plugin which sends information of each API call to a TCP server. It logs everything about API call, except the bodies of request and response. We realized that having bodies of request and response would be a great help for our engineers to trace various kinds of issues. But the kong logger plugin does not support logging bodies of requests, responses. So I decided to write a plugin that logs body also.

In this post, I will talk about how we built that plugin and overview of the code. The code is available on github.

arvind-internet-omuni/kong-tcp-body-log

Before I give details about the plugin, let me give details of how kong works.

How Kong Works

Kong is built on top of Nginx. Nginx is the most popular reverse proxy and load balancer out there. It powers almost 29% of top 1 million busiest sites of the world. Nginx was built in 2004. Nginx later added capability to embed lua (a programming language) code into Nginx, which brought tonnes of new opportunities and possibilities into Nginx as one could build applications on core functionalities of Nginx.

OpenResty is one such platform built on top of Nginx. It bundles many Nginx modules and Lua libraries. OpenResty Lua libraries execute directly in an Nginx worker. Kong is an OpenResty application. Kong implements clustering, plug‑ins, and a RESTful API that you can use for managing the API gateway

Kong Architecture

Kong Plugins

The most powerful capability of kong is that it can execute a plugin before it proxies the requests to upstream service. A kong plugin is a pieces of code (in lua) that is executed before calling the upstream API.

There are several kong plugins available. For examples — Plugins for traffic control, security, monitoring, logging etc. Browse a complete list of kong plugins here.

Anatomy of a kong Plugin

A kong plugin consists of two main files:

• Handler — In this file you specify the custom code to execute at different times in the lifecycle of a request. e.g. You may want to execute code when request arrives from client or when response is received from upstream server. In Handler you specify the code to be run at various such contexts. A good intro to handler can be found here.
• Schema — Here you specify the configuration for the plugin. For example in our case we wanted to configure the TCP end point for our plugin.

Implementing our TCP log plugin

The task was to log request, response bodies and headers to a TCP server. The default tcp log plugin from Kong does this except logging the request, response bodies. So I took TCP log plugin and extended it to log bodies as well. This was tricky as I had to change the default serializer such that it can also de-serialize the bodies. I looked at runscope plugin’s code to understand how it extracts body of the request. In the following section, I will give high level walk through of the code.

Code Overview

The entire code is on github. The instruction to install and run this plugin is also there. In this section I will walk you through the code.

Entire plugin is implemented in 3 files — handler.lua, schema.lua, log.lua.

handler.lua

• When a request is received from a client, function access() is executed. It captures the request body.
• Similarly when response for a request is received, then function body_filter() is invoked. It captures the response body.
• In both the cases respective headers are also captured.
• Once the bodies and headers are captured, it prepares a message, wrapping bodies and headers and then calls the TCP log() function of log.lua which sends this information over TCP.
• There is one important thing to note here —In Kong request and response are asynchronous operations, then how request and responses bodies are shared? Nginx provides a way to share data between different phases of same request using ngx.ctx. It is a lua table, that stores per-request Lua context data and has a life time identical to the current request. You can read more about it here.

log.lua

• Connects to TCP server and send the body in the payload
• log() functions is called from handler to log the request
• Maintains keep alive and timeout

schema.lua

• Accepts tcp server end points, keep alive, timeout configuration

That’s it! This plugin can also be extended to log to http end points instead of TCP. You just need to change the log.lua file to write to http end points. The http-log plugin of kong will be a good reference.

Thanks you for reading.

Cheers!