Performance testing with Postman

Postman is undeniably my first choice when I need to test any APIs. It is a tool that constantly evolves and offers so many more features than you would think at first sight.

So when I came across the opportunity to learn more about chatbots, I was wondering which framework to use as a couple of options are available out there such as DialogFlow or Amazon Lex to just name the most popular.

To help me define which Natural Language Processing (NLP) was performing the best, I decided to leverage Postman which you will see allows running hundreds of different cases if you wish to.

To do so, we will need:

• a basic chatbot for which we want to test the NLP performance.
• a node application that will receive the results from Postman and write them in a CSV file.
• a Postman client where all the magic will happen.

As a user, you will be interacting with Postman client and our performance test will be divided into 3 steps:

• Step 1: you are going to call the DialogFlow API from Postman and get a response that contains the information that you are looking for.
• Step 2: Postman, as part of a post-execution logic is going to call a local Node application that will process the given data
• Step 3: The relevant data are getting written into a CSV file which can then be analyzed by yourself.

Build the simplest chatbot possible

For this example, I decided to go with DialogFlow. Very quickly, you can build a simple chatbot that will understand “hello” and “goodbye”.

Visit DialogFlow and either create an account or sign in if you already have one.

On the left-hand side, you can create a new bot by clicking on the dropdown arrow and select “Create a new agent”

In the central part of the screen, fill in the agent name at the top (GreetingsBot in my case) and click on create. Wait for the training of your agent to be completed. It will take a very short time and you will be notified in the bottom right corner of your screen when this is complete.

Then, on the right-hand side, you can test your bot by typing “hello” and see that the bot recognized the entry as the default welcome intent. You can then click on “COPY CURL” to retrieve all the information required to connect to your DialogFlow bot.

Call DialogFlow API from Postman

From the information, you copied from the curl, you have the URL, the bearer token, and a body sample. Add your URL as a POST one and from the “Authorization” tab, select “Bearer Token” in the dropdown, then paste your token from the curl into the “Token” field on the right.

In the body tab, add the corresponding body that you got.

{
   "queryInput": {
      "text": {
         "text": "hello",
         "languageCode": "en"
      }
   },
   "queryParams": {
      "timeZone": "America/Toronto"
   }
}

You can now send your request and get a 200 OK response with the corresponding body response to your call.

Create a new collection that you name “DialogFlow” from the left-hand side of your Postman client and save the above request that you build into this collection.

Build the NodeJs application

The application is pretty simple. It runs locally and you just need a few packages to run it. You need to have npm installed then you can run the following commands:

npm install express fs body-parser csv-writer
mkdir ~/nodeApp
cd ~/nodeApp
touch script.js

You can then go on and open the script.js file that you just created into your nodeApp folder. You can copy and paste the following code in it:

const express = require('express');
const fs = require('fs');
const bodyParser = require('body-parser');
const createCsvWriter = require('csv-writer').createObjectCsvWriter;
const csvWriter = createCsvWriter({
   path: './performanceReport.csv',
   append: true,
   header: [
      {id: 'text', title: 'TEXT'},
      {id: 'executiontime', title: 'EXECUTION TIME (in ms)'},
      {id: 'intentmatch', title: 'INTENT MATCH'}
   ]
});
const app = express();
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json()); // Body parser use JSON data
app.post('/launches', function(req, res) {
   const records = [
      req.body
   ];
   csvWriter.writeRecords(records).then(() => {
      res.send('Saved');
   });
});
var port = 3000;
app.listen(port);
console.log('Express started on port %d ...', port);

Save the file and now you can start the application from your command line terminal:

node script.js

As you can see from the script code, it will write the results into a CSV file in the same folder as your script in a file named “performanceReport.csv”. It will contain 3 columns:

• text: the text given as an entry
• executionTime: the time it took to process the text by the NLP
• intentMatch: giving the intent found by the NLP

Let’s get to Postman

Now, we have everything we need to run our performance tests. To describe a little more the scenario, we want to send a list of possible phrases that a user can use with the chatbot and see how the NLP is performing by returning for each entry the intent mapped and the execution time for the query.

So we want to run the same call multiple times with different inputs in the body request. And we ultimately want to write the results into a CSV file.

First, let’s build the list of phrases that we want to pass to Postman. It accepts both JSON and CSV. I decided to go ahead with CSV. You can create a csv file with the following content:

text
hello
start
hi
how are you today
good morning

For the CSV format, each column will match a parameter whose name is defined by the header value.

In this case, we will use just one parameter: “text”.

To use the values from this file, we need to leverage environment variables. Update the body of your Postman request as follows:

{
   "queryInput": {
      "text": {
         "text": "{{text}}",
         "languageCode": "en"
      }
   },
   "queryParams": {
      "timeZone": "America/Toronto"
   }
}

Then, in the “Pre-request Script” tab, add the following line:

pm.environment.set("text", pm.iterationData.get("text"));

This is setting the “text” environment variable before every execution based on the iteration data which are the data that will come from the CSV file.

After, go to the “Tests” tab and copy and paste the following code:

var jsonData = JSON.parse(responseBody);
var intent = jsonData.queryResult.intent.displayName
pm.sendRequest({
   url: 'http://localhost:3000/launches',
   method: 'POST',
   header: {
      'Accept': 'application/json',
      'Content-Type': 'application/json'
   },
   body: {
      mode: 'raw',
      raw: JSON.stringify({ text: pm.environment.get("text"), executiontime: pm.response.responseTime, intentmatch: intent })
   }
}, function (err, res) {
   console.log(res);
   console.log(err);
});

As you can see, we are retrieving the intent name from the request’s response. We then send a request to our locally deployed Node application. The body is built with the values we intend to have in the CSV results file:

• text: from the environment variable
• execution time: from the Postman response
• intentMatch: from the Postman response

Now, we are ready to run. In case you are getting an error 401, it means your token has expired and you need to go get a new one in DialogFlow using the “COPY CURL” button that we encountered at the beginning.

Select your collection and choose “Run”

It opens a new window for Collection Runner. From there, click on “Select file” and choose your input sample data file that contains our phrases to be tested by the NLP. You can click on “Preview” to make sure your data are loaded correctly. Then run your test. Everything should go smoothly without error and should see as many iterations as you had entries in your input file.

If you look at your project folder, you will now see the performanceResult.csv file. Open it and you can see the list of values from your input file as well as how much time it took to execute each of them and to which intent the NLP associated it.

Note that some of them default to the fallback intent. That means you need to train your bot and add more sentences for your bot to understand these phrases and match them correctly.

Conclusion

You learned how to run tests iteratively based on an input file (CSV or JSON) and to store the results of these executions in a CSV file using an external Node application. Running this test allowed us to define based on real data which NLP is performing best by comparing the results obtained.

The possibilities of Postman are often underexploited and I hope this small tutorial will inspire you to dig more into what this tool can offer.