Simple test data generator on top of Puppeteer

Generating test data is a common problem in software development projects. Test data are needed for different purposes:

• you want to provide a test system for users who should work with the system and it is always a good idea to have data already in the system
• you want to make real life tests with your system with many data sets to check performance and system behaviour

Creating test data can be achieved in many ways:

• manual input- you have someone who will create test data in your system by clicking around in the application and creating items by hand. Not ideal, but it works for a smaller amount of data.
• your system has a REST API to create data. You can use tools like postman to call your REST API. BUT: Sometimes systems or certain parts do not have an api — also in 2018.
• you directly create data in the backend database. Sometimes you do not have access to the database and also sometimes it is no good idea to create data in a complex database scheme where you have to create data in several tables in the correct order…

Automate by using your browser

How about the idea to let a browser do the work? A relatively new project to control a chromium/chrome browser instance is Puppeteer.

The library is developed by the Google Chrome developers. Puppeteer consists of a javascript api which can be used in node based projects.

GoogleChrome/puppeteer

We will start to develop a small example here which creates test data in a Jive backend. Jive is a social business platform and provides user a possibility to create blog posts, documents and discussion.

Configure your testscript generator

It is a good practice to configure parameters as input, so that you are able to reuse the testdata generator in certain environments with different urls and login parameters.

Later you can call the script and overwrite the default parameter values with your own parameters like another jive placeid.

JIVE_PLACEID=2000 node testdata.js

These are the default values in the script which will be used when they are not overwritten from outside.

let host = process.env.JIVE_URL || 'http://localhost:8080/sbc';
let placeId = process.env.JIVE_PLACEID || '2004';
let number = process.env.JIVE_NUMBER || 300;
let user = process.env.JIVE_USER || 'admin';
let password = process.env.JIVE_PASSWORD || 'admin';

Next step in the script is to get access to the puppeteer api by using require which uses the underlying javascript module system.

The lorem-ipsum library is used to generate test data for description and titles.

console.log("Start to create " + number + " sample documents in place " + placeId + " on jive system with url " + host+ " and user "+user);

const puppeteer = require('puppeteer');
var loremIpsum = require('lorem-ipsum');

console.log("Launch puppeteer");

Login into Jive

Jive provides a login dialog, so that the user is able to create content being authenticated successfully.

Ok, so we want to launch a chrome browser headless, load the login page and fill in the necessary data (username and password) into the login dialog.

We type the user and password into the input field of the login dialog with the help of puppeteers api.

Puppeteer will wait after the click to the submit button that the page after login has loaded completely and will proceed with the logic after that.

async () => {
    const browser = await puppeteer.launch({headless: true});
    console.log("Launched headless chrome");

    const page = await browser.newPage();
    await page.goto(host + '/login.jspa');

    await page.type('input[name=username]', user);
    await page.type('input[name=password]', password);
    await Promise.all([
        await page.click('input[type="submit"]'),
        page.waitForNavigation({waitUntil: 'networkidle0'})
    ]);

    console.log("Login to development system successfull");

Fill html forms

After successful login we will now be able to create test data in the system by using a form page and fill in the needed data. The following logic will be executed in a loop.

• navigate to a certain page with http parameters

await page.goto(host + '/create-item!input.jspa?containerType=14&containerID=' + placeId);

• generate a title with an loremIpsum generator and type the value into the title html field

let title = loremIpsum({units: 'words', 
count: Math.floor((Math.random() * 20) + 1)});
await page.type('#title', title);

• Generate a random price and type it into the price field

await page.type('#price', 
Math.floor((Math.random() * 100) + 1).toString()); 

The form is filled now correctly and we want to create the content in Jive now. Jive provides a button with the css-class j-btn-callout to send a HTTP POST to the backend.

await page.click('.j-btn-callout');
console.log("Created content " + i + "/" + contentNumber);

The script runs and creates the number of items in the application.

Summary

This post should show a another possibility to generate test data. Puppeteer provides a nice and powerful api which is easy to learn. It allows to generate test data very easily…