Testing Payments: Not just the Test Cases

Photo by Jonas Leupe on Unsplash

The FinTech industry is a huge one and continues growing year over year. We see a lot of new software and companies in this sphere: from Neo Banks (like Chime, Current, MonoBank, etc.) to completely new Payments ways, like BNPL.

“Would you like to join the Payments team to help us test the product?” I sometimes ask the QA community.

“Nope, thanks. The domain is kinda “scary” since it has to do with the real money and there is a lack of information out there that would help QA build the testing strategy step by step”, I often hear in response.

So today, in this article, I’d like to address both of those concerns.

How do you check the quality of a Payment software? What are the main things to consider in order to start the journey of testing this kind of software? Will you need to share your personal Credit Card information for testing purposes?

In this article, I’m going to share with you my personal experience of testing the integration of PSP (Payment Service Provider). We truly learned a lot while building the successful Payments by WIX Platform, which currently serves millions of businesses worldwide.

Disclaimer: the purpose of this article is not to cover all the cases and give you a step-by-step guide (since the domain is really huge), but to give a more wide overview of things you should consider prior to creating your Test Cases.

First of all, in order to better understand Payments in general, let’s start with some terminology.

Terms

Payment Service Provider (PSP)

A payment company that provides merchants with a set of online tools and API(s) to accept, monitor and control payments. Acting as the gateway for the many payment method types, they usually work as SaaS with a revenue share business model. Here’s some further reading on the subject.

API

API stands for application programming interface. It allows two applications to communicate with one another to access data they need. Every action you take on your phone or Desktop browser, like sending a direct message or checking the likes on Instagram, uses an API to access and deliver that information (more about API in simple words can be found here).

Sandbox

Put simply, a Sandbox is an environment that testers can use to mimic the characteristics of a production environment and create simulated responses from all APIs an application relies on. Stripe and PayPal are examples of well-known PSPs that provide developers and testers with feature-full and robust Sandbox environments.

Testing Stages

Gathering requirements. API overview.

So, before you start testing the integration by creating your Test Scenarios, make sure you’re getting sufficient information about the integration itself, especially third parties. You have to clearly understand which APIs are being integrated with your application.

It’s also good practice for QAs to encourage developers to show you (or create one if it’s missing) a Time Sequence UML Diagram, which showcases how the system that’s actually being tested uses 3rd party API(s) and clearly visualizes the user scenarios. That makes it easier for QA to come up with real Test Cases afterward.

Why is that so important for you as a QA?

Checking the API references will provide you with a better overall picture of the integration.

• You will be able to understand what exactly has to be developed and how. You’ll have guides and examples of Components / APIs which will be integrated with your service;
• You might find gaps the requirements provided by the product;
• You might face the fact that an incorrect version of an API was integrated, for instance;
• And most importantly, you’ll be able to ask the right questions about the integration in general and build your test plan accordingly.

For a faster investigation in possible future production issues, a good habit is to store the version of 3rd party API(s) which were integrated somewhere in a knowledge base (since they might differ by time).

So, reading and understanding the API of the service you’re about to test is a must for a successful and fast release.

Also, shift left testing works here. The sooner QA is involved, the cheaper the fix would cost the company.

Manual Testing

It might sound obvious, but I’m going to say it anyway. It is your task as a QA to figure out what is the recommended way to test the integration from the PSP side.

When it comes to testing the PSP integration, usually you’ll be able to find the Testing Your integration Page on the PSP’s website which would explain the way the testing might / should be done from their perspective. Here, it’s important to focus on verifying the integration scenarios between two systems and to not end up testing the 3rd party API (even though, unfortunately, you might face some issues on their side as well).

What’s also important for you as a QA in this stage? You have to make sure that the development team thought about testing and how it should be done. The worst-case scenario at this stage can happen when the product is almost ready, but nobody knows how it might be checked prior to the release.

So the rule of thumb for our team is usually to collect the questions about the future testing of a feature, groom it, plan it, and put it into the ETA of delivery. In Payments by WIX, together with the development team, we have created the Architecture Design Template which includes the Testing part — so that we’re thinking about it during the planning phase. It is true that we might need to develop a particular tool for QA in order to be able to check a feature. And so that too should be a part of the planning phase.

So, back to PSPs.

There are several scenarios that usually allow you to check an integration without touching the live data (and money): using test mode/keys or using the PSP Sandbox API. This addresses the first concern of QAs: “should I use my personal Credit Card in order to Test the integration?” Nope (at least not always) :)

Test Mode / Keys.

This approach allows you to perform testing in a Test Mode (without the need to complete the live account verification) and use all of the features of the same API, just with different parameters.

A PSP that’s a good example of this is Stripe. Here’s their recommendation for testing:

“You can use the Stripe API in test mode, which does not affect your live data or interact with the banking networks. The API key you use to authenticate the request determines whether the request is live mode or test mode”.

But again, make sure your system supports it (meaning there is a way to connect/send the appropriate value from your application).

Sandbox API.

The most popular way to check the integration of a PSP is to use the Sandbox API. in an ideal world, theoretically, the Sandbox environment should be the mirror of the Production — which would allow you to test the integration without touching any live data.

Pros of using Sandbox:

1. ability to test the integration without the need of real accounts;
2. an easy way to simulate (and test) edge cases (in our case different transaction statuses);
3. ability way to build sufficient E2E automation coverage;

Cons of using Sandbox:

1. environment / API might not be stable;
2. might behave differently than the production / live environment;
3. might be outdated;
4. might be down (while live is up), which results in flakiness of your automation tests;
5. reporting the issues with Sandbox to the company that owns and operates it can be a pain (since it might be a “known issue” or “the way things work”);
6. missing newest features

PayPal Sandbox is a good example to play with, which includes both pros and cons of using it.

Production / Live Testing.

Is it enough to test the integration in just the Sandbox environment before enabling it for real users? Glad you asked!

Truth be told there is actually a third option of checking the integration — by doing the LIVE verification on production. Meaning, you would need to have a real, activated account on a PSP (as for your Merchant) and a real Credit/Debit Card (or Buyer account). This way you will be able to make sure that the integration behaves as expected. Yet that’s not always possible, to be quite honest. You might face some trouble with either opening a real account with that PSP (since you’d need to pass the KYC form) or you simply might not have access to a real Credit/Debit Card for testing.

Why we practice live verifications

• There is no way to check things inside a Sandbox (either legacy code or lack of SB support);
• The project is risky and we want to be 100% sure LIVE env works the same way it did when we tested in Sandbox;
• Production Issue with a real customer. In order to debug a particular case sometimes we need to perform a live transaction to simulate the case of a particular user.

So what’s the best strategy for the PSP integration check?

I’d suggest implementing the Sandbox support anyways, checking all the possible test scenarios against the SB, and, if that’s possible, doing the sanity check on production prior to the release. In this case, you will cover much more than just checking several cases against production (since you’ll stumble upon the limitations).

Communication

When it comes to integrating with third-party applications (in our case PSPs) one of the key factors influencing velocity would be communication: internal & external.

Internal communication

It’s possible that I’m now saying something that’s old news, but it’s worth noting anyway. Requirements. You have to make sure that product, developer, and QA working on the feature are all on the same page. You’ve groomed (massaged) the feature. Asked all relevant questions.

What do we practice in WIX?

• Pair testing with QA & Developer.
  This will provide a velocity boost, especially when it comes to Back End testing (e.g. server to server integration cases). QA knows the product better and the server developer knows the most efficient way of checking the edge cases. Thus, together they achieve more. As a side effect, both QA & BED benefit, because developers get more knowledge of the product and QAs get more useful ways/tools of checking the API/server.
• You own what you do.
  Since we have a big QAs to Devs ratio (1 to ~10) and because there is a shared product quality ownership, the rule of thumb in our team is to be responsible for what you do. That means developers are checking their changes prior to passing them to QA. Not only by adding unit tests, but by passing some basic happy flows of their changes. Thus, QAs are more focused on progression rather than regression testing.
• Checklist feedback sessions.
  The stakeholders of the feature (QA, Developer and Product) sit together to review the checklist put together by the QA and to brainstorm / fulfil all possible scenarios which are not in the list at the moment. As a result we get three pairs of eyes and three brains instead of just one covering the feature from different angles, which is super valuable in such a risky domain as Payments. Visibility is a great side effect of this exercise. After the meeting, all of the stakeholders should be aware of what exactly is planned to be tested.

External communication.

Since we’re talking about the integration between two companies, good established internal communication is usually 50% of the success, since we have external dependencies.

The R&D team must have a contact person from the PSP company to ask any integration-related questions. Quite often we run into situations where the documentation is not clear enough or out of date or even missing. And if you do not have an easily accessible person on the other side, you might be blocked for a while with your open questions.

What do we practice in WIX?

Since we have a long list of Providers (PSPs) already integrated into our Payments platform (more than 60 when this article was written), we have dedicated shared Slack channels between WIX & PSPs where both sides can raise questions and get those answered rather quickly. Of course, sometimes there are delays in responses, so this allows us to put the right priorities and build an efficient queue of integrations.

For you, as a QA these communication channels are super helpful when you test the integration.

It might be true that your company is not as big as WIX so having close communication with a PSP might not be possible. In that case, make sure you have at least a verified technical support channel (like email) to forward your questions. Usually, this is the task for the Product Manager.

Automation

So, you’ve tested the integration manually, it passed QA and you’re about to release it. Are you sure you’ve done enough for the project? Because the answer might be no. In addition to the manual and exploratory testing, you should also build automation coverage for the project. You have to make sure that the integration is alive at any given moment and cover the regression on your side in order to get rid of manual repetitive work.

“Automate what can be automated” is our rule of thumb at Payments. In the next articles, I’ll share how our automation model looks for the Payment platform, which coverage we have on UI and API level, and a lot more. Stay tuned.

Conclusion.

I hope with the help of this article you were able to get a glimpse into what needs to be done in order to test the integration of your application with any PSP. Now you understand that the Test Cases are not the only thing to be considered while preparing your testing. Understanding API, implementing the Sandbox, building the communication channels are also key ingredients to your successful release.

Upcoming articles of Testing Payments topic will probably include the test plan example, tools QAs use daily, production monitoring approach, collaboration with support, processes, automation approach, and a lot more. So, bear with me. I’ll be happy to read your comments and will try to answer any questions you may have.