Wiiisdom Labs
Published in

Wiiisdom Labs

Cost-Efficient Integration Testing on vendor appliances running on EC2

“person holding pencil near laptop computer” by Helloquence on Unsplash

We are building applications that are relying on monolithic commercial applications like SAP Business Objects or Tableau Software. Following our quality standard, we want to run reliable, efficient and cost-effective tests to validate our work. Here is how we are doing that using AWS EC2 instances.

The need

Integration Testing and Acceptance Testing are necessary steps to insure quality in our products. But this kind of tests needs to interact with existing vendor solutions, not available in an SaaS model. For example we are working a lot around SAP BusinessObjects and we need to run tests against this kind of software.

Integration Tests relying on external systems

The first idea

An easy solution is to deploy an integration server, secure it to deny everyone the possibility to modify the content (it needs to stay stable since tests are relying on the content), and run it all time. Do that for example in your existing local infrastructure or drop that on a EC2 at AWS. This is a simple solution but it might be very costly (you only need it a couple of minutes a day ! ) and because content need to be the same you probably want to reset it at its original state before performing additional tests. You pay around 143 US$ for a t2.xlarge when it’s up all time.

The solution

Because our build process is managed by Maven we are searching for a Maven plugin solution for that. And lo and behold it exists !

We have decided to fork an existing project and adapt it to our specific needs. The plugin is able to create an EC2 instance on-demand based on an existing image when the integration test goal is launched and terminates it at the end of the goal. The customization allows us to wait until the vendor application is fully started on the EC2 (by listening to TCP ports).

It doesn’t rely on an existing EC2 instance but on a Amazon Machine Image (AMI) and this is good news :

  • each integration test ran is done on the exact same AMI each time, we can rely on the server's content being absolutely consistent through time.
  • multiple integration tests can be launched in parallel (2 developers, jenkins server…), each one will create a specific instance.
Process of the Maven Plugin actions

The project integration is very simple. You can call for the proper maven plugin in the pom.xml file :


The execution and terminate goals are linked to pre-integration-test and post-integration-test phase of the classic Maven Failsafe plugin.

To use the AWS DNS entry we use the configuration of failsafe to set a System Property variable that we can use in our code:


We now have a robust integration test system that rely on AWS, with a very low cost (around 30US$ per month for a 10-person project).


There is still some room for improvement. A few ideas that we have:

  • Replace the failsafe plugin by something else. Currently when a test fail then the post-integration-test step is not trigged so the EC2 instance is not terminated.
  • Optimize the build when we build a multi-module Maven project (currently each Maven module starts an independent instance and we‘re losing time when multiple modules are running IT tests).

Interested by the subject ? Don’t hesitate to comment below or follow us ! We at GB&Smith are building top-notch tools for BI products in our French and Canadian offices.




Content produced by engineers working at Wiiisdom.

Recommended from Medium

Creating A Cooldown System in Unity

Impressions of SharePoint Saturday Barcelona 2017

Emphasise on “Logic” not the “Language”

Ontology & ApeSwap Bring it Back 🌴

Introduction to GraphQL

How HyScale’s App-Centric Abstraction helps solve Kubernetes complexities

Kubernetes and containerization helps in cost optimization, scalability, multi-cloud benefits with no vendor lock-in

Two Heaps; Min Heap & Max Heap

Lives and Respawn System in Unity

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Julien Bras

Julien Bras

Innovation Team Leader @ Wiiisdom. Love testing & using new things. Dad of 3.

More from Medium

Peculiar AWS CloudFormation failure

An introduction to managing your API with Amazon’s API gateway & custom authorizers

Amazon Rekognition for Identity Verification

Internal domains with AWS Route53 Private Hosted Zone — Part 1