⚙ Integration tests on Node.js CLI: Part 2—Testing interaction / User input

This article is part of a series about writing Node.js CLI programs and more specifically, testing them by writing E2E/Integration tests, rather than writing the usual module unit test. If you want to skip to the final implementation code, check here. The links to the other parts are listed below:

• Part 1: Why and how?
• Part 2: Testing interaction / User input
• Part 3: Inter Process Communication
• Part 4: Mocking Services

Most CLI tools are composed of some command input and an expected output, and that's fine for most of them. However, as more developers jump on the CLI bandwagon, they expect their interfaces to provide nicer means of communication without leaving the "comfort" of the terminal window. Being able to pick options from a list, answer questions and provide input in context is a huge win when discovering the tool. As we switch gears into this new — or should I say, terribly old but trendy once again — form of User Interface, we discovered that some of the design principles we apply to products we interact on a daily basis such as web and mobile apps, principles like user-friendliness and well thought communication, are also a requirement for these tools. Such principles apply of course to Command Line Interfaces, but also to other non-graphical interfaces like bots or AI assistants that are slowly taking over the world (in a totally non-Skynet way) becoming part of our daily lives. This translates into investing time and effort in delivering a quality product, which inevitably leads us to automated testing.

In Part 1 we explored how using Node.js as our platform to create CLI tools, we can achieve some form of E2E test automation that extends beyond the usual (but not any less important) unit tests. In this entry we will add up to the complexity by testing user input to deliver consistent UI paths and clean UX while providing means to our users to interact with our tool in a friendlier way. I mean, the terminal is rough as it is for the newcomers, let's make sure they feel a bit less intimidated, right?

But first, answer me this

We used commander's pizza example in the last post to provide an easy entry point to our application logic. We'll now use the equally awesome inquirer module to handle user input. This module provides some really nice and clean interfaces, abstracting some of the most used forms of input that an user can expect in other graphic interfaces. Turns out they also have a pizza example of their own! Safety disclaimer again: we’re assuming the use of Node.js version ≥ 8, which includes async/await support. If you’re still not familiar with the syntax, check this awesome article about it. Let's order some pizza!

This is an example I/O of the pizza example in inquirer

inquirer now adds the ability to prompt the user for an answer, and will not end the process until either the user answers, or forces an exit (usually Ctrl + C) to end it. In order to write a test that suits the whole case of this command and answers all the questions, we may need to modify our previous function cmd.execute to support the ability to pass inputs. The test will look somewhat like this:

// Pizza CLI test: User Input Take 1
const expect = require('chai').expect;
const cmd = require('./cmd');
const { EOL } = require('os');
describe('The pizza CLI', () => {
  it('should print the correct output', async () => {
    const response = await cmd.execute(
      'path/to/process',
      [],
      [
        'y',
        '555-1234123',
        'Large',
        '1',
        'p',
        '2',
        'My Comment'
      ]
    );
    expect(
      response
        .trim()
        .split(EOL)
        .pop() // Get the last line
    ).to.match(/^Order receipt/); // Using chai-match plugin
  });
});

The second array will provide the simulated user inputs. But how is it supposed to simulate the inputs? Turns out that we're feeding them to the stdin of our child process. So we'll modify our execute method to support this feature:

const concat = require('concat-stream');
function executeWithInput(
  processPath,
  args = [],
  inputs = [],
  opts = {}
) {
  // Handle case if user decides not to pass input data
  // A.k.a. backwards compatibility
  if (!Array.isArray(inputs)) {
    opts = inputs;
    inputs = [];
  }

  const { env = null, timeout = 100 } = opts;
  const childProcess = createProcess(processPath, args, env);
  childProcess.stdin.setEncoding('utf-8');
  
  let currentInputTimeout;
  // Creates a loop to feed user inputs to the child process
  // in order to get results from the tool
  // This code is heavily inspired (if not blantantly copied)
  // from inquirer-test package
  const loop = inputs => {
    if (!inputs.length) {
      childProcess.stdin.end();
      return;
    }

    currentInputTimeout = setTimeout(() => {
      childProcess.stdin.write(inputs[0]);
      loop(inputs.slice(1));
    }, timeout);
  };
  const promise = new Promise((resolve, reject) => {
    childProcess.stderr.once('data', err => {
      // If childProcess errors out, stop all
      // the pending inputs if any
      childProcess.stdin.end();

      if (currentInputTimeout) {
        clearTimeout(currentInputTimeout);
        inputs = [];
      }

      reject(err.toString());
    });
    childProcess.on('error', reject);
    // Kick off the process
    loop(inputs);
    childProcess.stdout.pipe(
      concat(result => {
        resolve(result.toString());
      })
    );
  });
  return promise;
}
module.exports = { execute: executeWithInput };

Ookay. So what's going on here?

Feed me!

With the addition of inquirer, we added a module that, like I said above, will not end the process until either the user answers, or forces an exit. This means that when we run the execute command, it will hang until one of these conditions are met. We are taking advantage of this condition and we will feed the inputs in chunks, using childProcess.stdin.write(). We'll need to give the CLI some room to respond — that's why we placed the timeout between each input — and here's a gotcha: this time will differ, depending on several conditions of the environment you're running the tests in, like machine resources (processor, RAM), other processes running at the same time, and of course if the shell being used has plugins and other software running on top of it. In any case, it seems to behave like the real thing. But not quite: if we try to run this code as it is, it will hang after the first input. Why?

There are steps that are so obvious we often forget they are there. Typically, when you type a command in the CLI you think:

$ mycommand -> answer questions -> done

But what you're really doing is:

$ mycommand ENTER
$ answer question 1 ENTER
$ answer question ..n ENTER
$ done

We need to feed the tool these commands too, because, after all, these are user inputs too, right? And that includes UP/DOWN arrow keys, SPACE, ENTER, and any other keys involved and supported by inquirer in their interface. So looking at our test case again, the inputs we send should look somewhat like this:

// Pizza CLI test: User Input Take 2
const expect = require('chai').expect;
const cmd, { ENTER } = require('./cmd');
const { EOL } = require('os');
describe('The pizza CLI', () => {
  it('should print the correct output', async () => {
    const response = await cmd.execute(
      'path/to/process',
      [],
      [
        'y',
        ENTER,
        '555-1234123',
        ENTER,
        'Large',
        ENTER,
        '1',
        ENTER,
        'p',
        ENTER,
        '2',
        ENTER,
        'My Comment'
        ENTER,
      ]
    );
expect(
      response
        .trim()
        .split(EOL)
        .pop() // Get the last line
    ).to.match(/^Order receipt/); // Using chai-match plugin
  });
});

If you're working in Unix based systems, you're in luck: turns out that each key has a unicode representation that if passed to stdin, will behave as if the user had pressed the key and had interacted with the CLI (the unicode characters are here, in case you're wondering). If you're on Windows, though, you won't be able to work these tests out. The reason is that Windows CMD and PowerShell don't have a string representation of the keys that we just mentioned, and having no string representation, they can't be fed to the child process. If you are reading and know a way to do this in Windows, please leave a comment below. I'm looking for a way to make it work!

What we have so far

We’ve updated our spawn process runner to support user inputs in the form of an array. We updated the tests and identified that user not only does string input, but also presses keys to select/interact with the tool. We identified common gotchas and found ways around some of them. This code is far from perfect, but it's a starting point to create successful E2E tests for a CLI application.

In part 3, we’ll talk about how to pass data between the test process (parent) and the child process, in a pattern defined as Inter Process Communication (IPC). If you feel really curious about the final implementation, check the gist here. I know it took me a while to put this post together. I mostly write code, so writing a blog post is quite a challenge. Many thanks once again to Dan for his review. See you next time!