We decided to build a robot which tests our whole solution by autonomously operating a card reader coupled with our SDKs.
Paying for something with a card is easy and convenient. It is crucial that card acceptance works smoothly at the Point of Sale. It is annoying to both the shopper and the merchant when things don’t work smoothly. Therefore, testing is very important to everyone here at payworks. We test our solutions exhaustively to make sure that all the transactions are behaving as expected.
However, thorough testing takes a lot of time : time we would rather use to build new features and improve our product. So, we built a robot that does some of the testing for us. Whenever we commit a new version to our continuous integration system, our robot springs to life and tests the builds.
Manual testing of credit card solutions is no fun
In the card payment industry, testing payment solutions is a manual and repetitious process. It usually involves a large deck of testing cards, numerous test scenarios and a person carefully going through all of them one by one. Popular examples for such testing suites are ADVT (Visa) and M-TIP (MasterCard). This repetitious process takes time and manual testing is prone to human errors.
Card emulation hardware like the Collis SmartLink Box makes our lives easier by emulating a variety of different cards and providing valuable feedback to the tester. The tester doesn’t have to fiddle around with a hundred test cards which are easily misplaced and difficult to keep organised.
The Collis software is connected to our internal testing applications (with the Android and iOS SDKs) through a TCP interface. We have also formalised the test cases such that our tester app automatically selects the right card emulation for the particular test case.
This makes testing a lot faster, but still involves the human component for inserting/removing the card and entering a PIN into the card reader.
Our new colleague is a robot and does repetitious work
To completely remove the human component to testing, we decided to build a solution that can automate that last remaining steps. Quickly a LEGO MINDSTORMS EV3 system was chosen as the construction system.
The core component of the EV3 system is in fact a small 300 MHz ARM9 processor. With the LeJos firmware installed, it becomes a powerful Linux machine, allowing SSH and execution of Java programs. LeJos comes with a Java SDK that makes developing software on the EV3 brick fast and easy.
The robot’s brick runs a small web service based on NanoHTTPD. Our testing app connects to it via HTTP and issues commands such as POST /entryPin/1234. The software then employs its sensors and motors for entering the PIN into the device.
It took us several iterations to come up with a final construction that is both reliable and fast. We use touch sensors for calibrating the motors during the initial start-up. Some parts needed extra fixation and are secured using cable ties to keep it steady. For performing accurate movements, we employed new linear actuators (the grey cylinders next to the left touch sensor). In general, we were amazed by the motors’ power and precision.
The current solution can run through all supported test cases (Chip and PIN) without human interaction. Test cases involving swiping (Magstripe) are not possible (yet). Each test case takes around 50 seconds, hence finishing ADVT and MTIP tests in just about half an hour.
We plan on extending the robot to handle contactless(NFC) test cases in the future, but for now, we are happy to let the machines take over some of our important tasks.