How to Automate BDD Testing in OutSystems, Part 3: Data-Driven API Tests With the BDDFramework

This is an introduction to building data-driven tests using the BDDFramework OutSystems Forge component. It’s the final chapter of a three-part series into testing OutSystems applications using the BDDFramework component.

If you’re just getting started with Behavior-Driven Development (BDD) testing, and you haven’t used the BDDFramework component, it’s a good idea to read part 1 of the series, an introduction to BDDFramework. In that article, we go over a few examples of how to test a server action in an OutSystems application.

For data-driven testing, we’ll be doing something different — testing a public API — to showcase how the framework can be used for such a scenario. Note that data-driven testing can be applied to server actions, and it is a perfectly valid type of testing in that context as well.

Step 1: Design the Test Scenario

The public API we’ll be targeting in these tests is restcountries.eu. It’s a very simple public RESTful API where you can obtain data about countries around the world.

For the purpose of this example, we’ll be using a very simple test scenario where we ask for data about specific countries and validate if the country capital is correct. To use it, we just have to make a GET call to the following endpoint:

https://restcountries.eu/rest/v2/name/{name}

Let’s say for instance that we use “Portugal” as the {name} parameter:

https://restcountries.eu/rest/v2/name/Portugal

After you make your GET call, you should receive a JSON response that looks like this:

{  
      "name":"Portugal",
      "topLevelDomain":[...],
      "alpha2Code":"PT",
      "alpha3Code":"PRT",
      "callingCodes":[...],
      "capital":"Lisbon",
      "altSpellings":[...],
      "region":"Europe",
      "subregion":"Southern Europe",
      "population":10374822,
      ...
   }
]

One of the output fields is the “capital”, Lisbon.

If we wanted to specify a test for this specific interaction using the Gherkin-syntax, it would be a very simple scenario:

Scenario: Obtaining the country’s capital
Given:
That there is a country called Portugal
When:
I request the data about Portugal
Then:
The city Lisbon should be outputted as its capital

To implement data-driven testing for this scenario we use the following approach:

Describe the scenario outline using Gherkin syntax parameters (in this case <Country> and <Capital>).

Specify the examples that we will test. The test scenario can be run repeatedly using different values for <Country> and <Capital> (for instance: Portugal, Lisbon).

Scenario: Obtaining the country’s capital
Given:
That there is a country called <Country>
When:
I request the data about <Country>
Then:
The city <Capital> should be outputted as its capital
Examples:

|<Country>   |<Capital>   |

|Portugal    |Lisbon      |
|Argentina   |Buenos Aires|
|France      |Paris       |
|Spain       |Madrid      |
|Canada      |Ottawa      |
|Nigeria     |Abuja       |
|Japan       |Tokyo       |

Step 2: Implement the Scenario Outline

It’s time to implement the Country and Capital scenario outline, using the BDDFramework. We create the BDDScenario inside a web block, implementing the initial “Portugal” test. The BDDScenario is now a scenario outline we can use as a template for further tests.

The groups of steps are the same as defined in the previous section.

The first step is implemented as an empty action — its purpose is only to respect the integrity of the Gherkin specification. We could have used another API or database to validate that the country is, in fact, a valid one, but it’s fit for the purpose of this example.

In the When step, we call the API we’ve consumed inside this test eSpace: restcountries.eu, requesting the data for the country “Portugal”. You can learn more about consuming a REST API in the knowledge base article.

We store the output of the API call in a Response structure, that is inside the web block. We can use this to validate the Capital field in the final Then step:

The implementation is pretty straightforward, but now let’s turn this into a scenario outline. The first thing we want to do is to create two input parameters (Capital and Country) for the web block (TestCapital) where we created the BDDScenario.

We now replace all of the usages of “Portugal” and “Lisbon” in the test scenario by the corresponding input parameters. Inside the BDDScenario, we replace the Gherkin syntax terms with expressions that show us the Country or Capital that we’re using in the test.

In the When and Then step implementations we replace usages of “Portugal” and “Lisbon” with the parameters Country and Capital.

And now we have a reusable web block for testing different countries and capitals!

Step 3: Validate the Data-Driven Test

Let’s now drag that block into a web screen and define the parameters to have the same Portugal-Lisbon example working as we had before:

After we publish this eSpace and access the web screen running the scenario through a browser, we see that it is working correctly.

Step 4: Implement Data-Driven Tests for the Scenario Outline

So, finally, we can now easily set up a set of data-driven tests for this scenario outline by using Static Entities and Table Records! First, we define a static entity containing all of the examples we defined in the previous section.

Then, we create a web screen where Table Records is fed with all of the data from the Static Entity, showing the Scenario Outline web block for each row.

When we run this test suite screen in a browser, we see the Scenario Outline repeatedly executed for all the records in the Static Entity:

If one of the scenarios fails (for instance, we set the capital of Portugal to be New York), that specific test will fail, but the others will be executed, passing successfully. This sort of test design pattern is powerful when you want to use specification by example at scale. You can use it to both describe behavior and comprehensively test your system. So go ahead and download the BDDFramework OutSystems Forge component, and please share your thoughts!