How to Join Batch Nodes in Kapacitor

In this article, you will learn how to easily join two batches to compute the difference between values from two different InfluxDB measurements (“Some kind of tables”).
The reason of why I’m writing this article is because in the official Kapacitor documentation I haven’t found how to do it. There are only examples with stream nodes (19/06/2018).

Figure 1: IoT and Kapacitor

Let’s use the figure above to explain it with a real use case. We have some forecasting algorithm that every 1st day of the month retrieves historical data and then predicts the next month speed data for a city road, and then we want to compute the difference with the incoming real-time speed values to measure our predictions performance.

Figure 2: Predicted data vs Real time Data

One thing that we can do it’s to think about using a Kapacitor stream node to capture the incoming real-time data, but it’s not a good idea because we also need to retrieve (query) the predicted data, and with the stream node we can’t do that. The stream node reacts only with real-time data that is continuously coming to the database; it doesn’t work with “pre-computed and saved” data like the predicted ones.

So let’s see how we can do it with two batch nodes. Here is the script for Kapacitor (you can also find it in this repository):

join_and_compute_error.tick

So, if you are thinking about using the stream node to capture the incoming real time values and then compute the difference with the predicted ones, it’s possible that you will need to make your own UDF agent node to query the predicted values. For this reason, and for simplicity, a recommend to use the batch nodes. Think that stream nodes are a subset of the batch nodes ;)

One way to check if we’re joining correctly our data is using Chronograph dashboards. As an example, I use a table and a chart to see if the join is correctly computed:

Figure 3: “Debugging” with Chronograph

I hope this article has been useful to you!

Useful info:

1. TICK Stack Sandbox to start playing with Kapacitor: https://github.com/influxdata/sandbox
2. Getting started with Kapacitor: https://docs.influxdata.com/kapacitor/v1.5/introduction/getting-started/