Greenhouse of test automation: part III
How to store TestCafé tests in an InfluxDB
🍹 A tester walks into a
TestCafé
and orders aMockito
.
Instead of reading bad jokes, wouldn’t you rather extract the data of TestCafé
tests and store them in an InfluxDB
? Do you want to visualize those tests on a Grafana
dashboard? Or are you looking for inspiration to create your own solution? Then you’ve come to the right place!
🌱 Welcome to the blog series
Greenhouse of test automation
; a journey of visualizing test automation. To build confidence in quality assurance through test automation, for the benefit of sustainable development. Dutch greenhouses excel in sustainability and serve as a main thread running through this journey.
This journey starts with the benefits of visualizing test automation. Followed by the concept of extracting test results, and storing them in an InfluxDB
, in order to make them visible on a Grafana
dashboard. This third article covers the implementation of this concept for TestCafé
tests.
How to extract and store TestCafé tests in an InfluxDB?
This was the big question of my quest to visualize TestCafé
tests on a Grafana dashboard. It led to a lot of obstacles, learnings and ultimately, the testcafe-reporter-influxdb
library. You can use it in the project where your TestCafé
tests are located, and it’s ALMOST plug & play:
- 🔌 Plug
- Fork thetestcafe-reporter-influxdb
library*
- You also need thedotenv
library to set environment variables - 🔧 Configure
- Specify theTestCafé
reporter
- Set the address of your Influx database (host, port etc) via environment variables - 🎮 Play
- Run yourInflux
database on the specified host and port
- Run yourTestCafé
tests
ALMOST…
Note that the testcafe-reporter-influxdb
library is generically set up for a project with TestCafé
tests run remotely via Gitlab
pipelines. There will be data specific to your context, so my advice:
*Fork the library and specify it to save the most relevant data for your project, like test types, categories, or application names.
Standardize the test metadata in the testcafe-reporter-influxdb
library like the example above shows. Then add it to your TestCafé
tests, like below. These tags will be saved, so you can filter on this data and setup Grafana
dashboards based on risk categories for example.
fixture`The user sees the home screen after a successful login`
.meta({ feature: Feature.LOGIN, risk: Risk.SMOKE })
Hmm.. on second thought not just plug & play🤔 No worries! I explain all usage details and possibilities in this README.
How does the testcafe-reporter-influxdb
library work?
This two-step concept of extracting test results and storing test results in an InfluxDB acts as a tool to explain the implementation.
Extract TestCafé test data
With the TestCafé
Reporter Plugin, it’s possible to tap into TestCafé
tests at different times during a test run. One of the built-in reporters collects data from these tests to generate a test report in XML
format. Unlucky for me, there is no reporter that out of the box stores test results in an InfluxDB
. However, the generator-testcafe-reporter
provides a template for a custom reporter.
In just a few steps the project is generated, and ready to implement the reporter methods. Reporter methods are called byTestCafé
during the test run; at the beginning and end of a test, fixture or complete test run. The available data can be extracted via the arguments. Within the testcafe-reporter-influxdb
the reporter methods are implemented to process and store this test data.
Store TestCafé test data in an InfluxDB
The example below shows a fragment of the implementation for reportTestDone
; the reporter method called when a TestCafé
test is done. I use the node-influx
client to interact with InfluxDB
in a Node.js
environment. This client helps to formulate, address and send the data to an InfluxDB
.
- ✉️ Formulate; set the data in the correct structure
reportTestDone
provides access to the data of one test;name
,testRunInfo
andtestMeta
. After extraction, the test data is saved in anIPoint
object, made available bynode-influx
. Apoint
stands for a single data record. It is linked to a table - or so-calledmeasurement
- and contains atimestamp
,fields
andtags
. Imagine sending anIPoint
object after every test to the database… that wouldn’t be efficient. Therefore the point is added to a collection of points, then reset for the next test.
- 📧 Address; set the properties for the correct database
node-influx
provides an envelope in the form of theInfluxDB
object to address the right database instance. Usingschema
, a validation is performed on the data that you send in that envelope. Are the fields filled with the correct field types? Like a lot of code, the schema is not forgiving. If one field is missing in the data, or does not contain the correct data type, this object will immediately give an error and the process will be aborted. I learned this the hard way…
- 📩 Send
At the end of the test run the methodreportTaskDone
is called. The moment where allpoints
are saved, collected and sent. Using theInfluxDB
object a connection is made to the correct database. Then allTestCafé
test data is sent via a REST POST call.
To be honest I wasn’t quite sure to start this adventure implementing the testcafe-reporter-influxdb
, but I would definitely recommend it. Especially if you aspire to learn more about automation. If you have a similar challenge to tackle, don’t be afraid, just go for it! The journey is as rewarding as the result!
What’s next? The implementation of extraction and storage of JUnit
tests in an InfluxDB
, and how to display this testdata in an Grafana
dashboard. Stay tuned!
🖇️ You are welcome to connect with me on LinkedIn, or follow me on Medium, to keep updated with my latest blogs.