Greenhouse of test automation: part II

Extract and store automated tests in an InfluxDB

🌱 Welcome to the blog series Greenhouse of test automation! A context-driven journey of visualizing test automation using InfluxDB and Grafana. Dutch greenhouses excel in sustainability and serve as a main thread running through this journey. Abstract concepts, implementation examples and learning points are provided to start your own journey.

Part I explains the concept greenhouse of test automation and two customer cases where this is applicable. The decision of visualizing test automation to build more confidence in quality assurance and to identify possible doubles/gaps in the test coverage. And the available tools in that context; InfluxDB and Grafana.

What do you need to extract results from automated tests? How can you store them in an Influx database ? Especially from different test automation tools? This second part answers all these questions and more. You will learn more about InfluxDB to store and visualize test results on a Grafana dashboard as efficiently as possible.

Extract test results

Extracting results from automated tests depends on the tooling you are using. I researched the test automation tooling used in the context of my client: Jest & TestCafé to test in the frontend, JUnit and Spring for the backend. For all this tooling it is possible to tap into the tests at different times during a test run. At the beginning and end of a test, test file (class/fixture) or complete test run. The available data depends on the tooling. Usually the obvious like the result of a test, its title, and any possible error messages are at your disposal.

A ‘reporter’ or ‘listener’ can extract several data of an automated test

So the concept is the same, only the name might differ.Reporters is a popular term for frontend test automation tooling, JUnit and Spring like the sound of listeners. If you use another tool, just try a Google search with your ‘tool name’ + reporter, listener or just output, you probably find one that does the trick for you as well!

Often standard reporters/listeners exist that export results in several formats, like xml. Unfortunately, there are none that out of the box store results in an Influx database. This means I have to create a custom reporter in JavaScript and listener in Java, what a challenge!

Store test results in an InfluxDB

Another challenge is to store the test results in an Influx database. Fortunately, there are several clients that allow interaction with Influx in different contexts. Below you will find a table with the available clients for JavaScript and Java. With InfluxDB version 1.8 it is recommended to use the new libraries supported by InfluxDB. Unfortunately, 1.7.9 is still used in my context, so the implementation examples in my future blogs for TestCafé and JUnit will refer to the older libraries.

check out https://docs.influxdata.com/influxdb/v2.0/tools/client-libraries/ for all supported contexts

Despite the different contexts and syntax, the concept of storing test results using the InfluxDB client boils down to the same three steps:

• ✉️ Formulate
  Set the data in the correct structure for InfluxDB
• 📧 Address
  Set the properties for the correct Influx database
• 📩 Send
  Send the data toInfluxDB via a REST POST call

The concept of storing automated test results in an InfluxDB

In order to efficiently store your test results in an Influx database and to visualize them effectively, it is useful to know more about Influx.

About InfluxDB

InfluxDB is an open source database and I can tell they like cool new terms. A database can consist of one or more measurements; a container of values comparable to a table. All measurements contain a specific column for time, because InfluxDB is a time-series database. This means that every database record is linked to a timestamp.

Other columns of a measurement can be filled with field or tag values. The main difference? Tags are indexed and fields are not, which has quite an impact on the query capabilities. Therefore it is important to consider how you want to visualize your test data.

If you want to display actual test data, like test titles, fields are great. However, if you want to filter or group on test results or other categories, tags are the best option for you. In the picture below you will find an overview of the differences between a field and tag, and some examples of appropriate test data.

The difference between fields and tags, and for which kind of test data they are most suitable

One single data record, or row of a measurement is called a point. A point is unique by the combination of a measurement name, tag set and timestamp.

🌱 NOTE: if a point is not unique, it will be overwritten! Quite a pitfall I fell deeply into…

Now you know more about the abstract concepts of extracting test results, and storing them in an InfluxDB. Also about how InfluxDB works, to retrieve and visualize results as effectively as possible. What’s next? A technical deep dive starting at the test automation tooling TestCafé and JUnit, to the Influx database, and to resurface again at the Grafana dashboard. Stay tuned for part three and find out!

🖇️ You are welcome to connect with me on LinkedIn, or follow me on Medium, to keep updated with my latest blogs.