Rich Data Visualizations with the New Relic Insights API
At Zoosk, we use New Relic to monitor our production traffic and user activity. The platform is incredibly useful for keeping a handle on our self-service deploys and the different parts of our microservice architecture. Not only is it useful, it’s incredibly flexible: the New Relic Insights API allows anyone to create their own dashboards and visualizations.
For a recent hack day I wanted to produce a real-time map of people connecting on Zoosk for the first time, as a way of demonstrating the amount of love that Zoosk brings to the world. In this post, I’ll be guiding you through how I did it.
Step 1: Find data in Insights
The first thing I needed to do was find the actual data in Insights. I did this using the Insights feature of New Relic, which allowed me to run queries against my data in an SQL dialect, NRQL. After some messing around with different possibilities, I came up with this:
As you can see, I’ve opted to ignore the regular results and instead selected the tab with curly braces. This lets me see the raw query response, which is exactly what I’ll get later when I call the Insights API directly.
Step 2: Get a key and call the Insights API
Check out New Relic’s documentation for how I set up my key. After I followed the steps there, I was able to start making queries using curl on the command line:
Robin$ curl -H "Accept: application/json" -H "X-Query-Key: ?" \
"https://insights-api.newrelic.com/v1/accounts/???/query?nrql=SELECT%20average%28duration%29%20FROM%20Transaction"Emboldened by this success, I moved on to building an incredibly light API client in Node (remember that this was for hack day, so some concessions were made for time).
Step 3: Set up a server with socket.io and express
When you’re trying to set up something real-time, socket.io and express are my go-to libraries. I can set up a fully-running server with real-time pubsubs in just two files. For brevity, I’m going to save you from having to read my express implementation since it’s the same as all the other tutorials out there.
The goal of the update code (again, hack day, expect liberties) is to continually ping the Insights API for updates (once a minute), and then publish the updates to all of the users that are currently connected to the socket. Once data is loaded, any users that connect are immediately served the cached data instead of needing to wait for new queries to be made. Since our Insights query includes timestamps, each of the clients will replay the data at the correct time.
Step 4: Set up a client using the Google Maps API
I wanted to plot my data on a map, and what better tool to use than the Google Maps API? It comes with all of the features I need out of the box except for clustering, for which a helpful tutorial is provided.
My client code actually ended up being a lot more complex than just drawing on the map, so I whittled it down to the core for my example map updater. This code shows off how to respond to events from socket.io, how to create map markers, how to cluster markers, and how to draw lines between points on the map. Luckily for us, everything else is taken care of by the API.
Step 5: Get creative!
Now that the Insights API was working, I took the opportunity to link up many other statistics. By simply adding more queries, I produced a live-updating interaction count display that we could put up in our office.
I highly recommend trying out the Insights API, whether it’s in JS or elsewhere. It can be used for automated reports, alerts, or even Slack bots! Get started here.
