Greenhouse of TestCafe automated tests

Greenhouse of test automation: part III

How to store TestCafé tests in an InfluxDB

🍹 A tester walks into a TestCafé and orders a Mockito.

Instead of reading bad jokes, wouldn’t you rather extract the data of TestCafé tests and store them in an InfluxDB? Do you want to visualize those tests on a Grafana dashboard? Or are you looking for inspiration to create your own solution? Then you’ve come to the right place!

🌱 Welcome to the blog series Greenhouse of test automation; a journey of visualizing test automation. To build confidence in quality assurance through test automation, for the benefit of sustainable development. Dutch greenhouses excel in sustainability and serve as a main thread running through this journey.

This journey starts with the benefits of visualizing test automation. Followed by the concept of extracting test results, and storing them in an InfluxDB, in order to make them visible on a Grafana dashboard. This third article covers the implementation of this concept for TestCafé tests.

How to extract and store TestCafé tests in an InfluxDB?

This was the big question of my quest to visualize TestCafé tests on a Grafana dashboard. It led to a lot of obstacles, learnings and ultimately, the testcafe-reporter-influxdb library. You can use it in the project where your TestCafé tests are located, and it’s ALMOST plug & play:

1. 🔌 Plug
   - Fork the testcafe-reporter-influxdb library*
   - You also need the dotenv library to set environment variables
2. 🔧 Configure
   - Specify the TestCafé reporter
   - Set the address of your Influx database (host, port etc) via environment variables
3. 🎮 Play
   - Run your Influx database on the specified host and port
   - Run your TestCafé tests

ALMOST…

Note that the testcafe-reporter-influxdb library is generically set up for a project with TestCafé tests run remotely via Gitlab pipelines. There will be data specific to your context, so my advice:

*Fork the library and specify it to save the most relevant data for your project, like test types, categories, or application names.

Specify the TestMetadata you want to use in your TestCafe test

Standardize the test metadata in the testcafe-reporter-influxdb library like the example above shows. Then add it to your TestCafé tests, like below. These tags will be saved, so you can filter on this data and setup Grafana dashboards based on risk categories for example.

fixture`The user sees the home screen after a successful login`
  .meta({ feature: Feature.LOGIN, risk: Risk.SMOKE })

Hmm.. on second thought not just plug & play🤔 No worries! I explain all usage details and possibilities in this README.

How does the testcafe-reporter-influxdb library work?

This two-step concept of extracting test results and storing test results in an InfluxDB acts as a tool to explain the implementation.

Visualization of the TestCafé test automation sensor: testcafe-reporter-influxdb

Extract TestCafé test data

With the TestCafé Reporter Plugin, it’s possible to tap into TestCafé tests at different times during a test run. One of the built-in reporters collects data from these tests to generate a test report in XML format. Unlucky for me, there is no reporter that out of the box stores test results in an InfluxDB. However, the generator-testcafe-reporter provides a template for a custom reporter.

In just a few steps the project is generated, and ready to implement the reporter methods. Reporter methods are called byTestCafé during the test run; at the beginning and end of a test, fixture or complete test run. The available data can be extracted via the arguments. Within the testcafe-reporter-influxdb the reporter methods are implemented to process and store this test data.

Store TestCafé test data in an InfluxDB

The example below shows a fragment of the implementation for reportTestDone; the reporter method called when a TestCafé test is done. I use the node-influx client to interact with InfluxDB in a Node.js environment. This client helps to formulate, address and send the data to an InfluxDB.

Formulate data from TestCafé tests in a point to fit in an InfluxDB

• ✉️ Formulate; set the data in the correct structure
  reportTestDone provides access to the data of one test; name, testRunInfo and testMeta. After extraction, the test data is saved in an IPoint object, made available by node-influx. A point stands for a single data record. It is linked to a table - or so-called measurement - and contains a timestamp, fields and tags. Imagine sending an IPoint object after every test to the database… that wouldn’t be efficient. Therefore the point is added to a collection of points, then reset for the next test.

Address the Influx database so that the TestCafé test data arrive at the right location

• 📧 Address; set the properties for the correct database
  node-influx provides an envelope in the form of the InfluxDB object to address the right database instance. Using schema, a validation is performed on the data that you send in that envelope. Are the fields filled with the correct field types? Like a lot of code, the schema is not forgiving. If one field is missing in the data, or does not contain the correct data type, this object will immediately give an error and the process will be aborted. I learned this the hard way…

At last! Time to send the collected TestCafé test data to the Influx database

• 📩 Send
  At the end of the test run the method reportTaskDone is called. The moment where allpoints are saved, collected and sent. Using the InfluxDB object a connection is made to the correct database. Then all TestCafé test data is sent via a REST POST call.

To be honest I wasn’t quite sure to start this adventure implementing the testcafe-reporter-influxdb, but I would definitely recommend it. Especially if you aspire to learn more about automation. If you have a similar challenge to tackle, don’t be afraid, just go for it! The journey is as rewarding as the result!

What’s next? The implementation of extraction and storage of JUnit tests in an InfluxDB, and how to display this testdata in an Grafana dashboard. Stay tuned!

🖇️ You are welcome to connect with me on LinkedIn, or follow me on Medium, to keep updated with my latest blogs.