What is reconciliation at HiPay?

Our mission at HiPay is to provide businesses with a powerful all-in-one payment platform. On top of it, we build a variety of financial features & products to help our customers conduct their operations. We also strive to provide our executives with strategic financial reporting & data.

In order to understand what reconciliation means at HiPay, we need some context.

We process payments for our merchants. How does it work? It’s simple: when a client wants to buy something on our merchant’s website (1), he is forwarded to our gateway (2) so that we can manage all the steps necessary to proceed with the payment (3).

But that is not the end of it. In fact, it’s just the beginning of the journey, since the money has to switch hands now.

First, the acquirer retrieves the money from the client’s bank account (1) and credits it to our bank account (minus his commission of course) (2). By doing so, he sends monetic information to HiPay informing it of all the transactions that were completed (3). When our bank statement confirms that the money has arrived on our bank account (4), we can pay our merchant(s)(minus our commission of course) (5) and send him a settlement report (6).

It’s at stages 3 and 4 that the reconciliation comes into play. The main goal of reconciling is to match 3 different data sources: HiPay transactions created by the merchant, monetic information from the acquirer and banking transactions. In the meantime, we extract fee informations from monetic data so that we can calculate our margin.

This process implies that we’re able to take care of multiple constraints such as:

• Different types of resources (sales, refunds, chargebacks, disputes, fundings, banking movements)
• Different amounts (partials or totals)
• Different currencies (client’s, provider’s and acquirer’s)
• Different means to retrieve monetic information (and there’s a lot of them ! !)
• Daily deadline to finish treatments in order to pay the merchants on time every day

How do we do all that?

Technically, the process is split into 3 parts : data acquisition, data normalization and the reconciliation itself. The first 2 steps are bound to the acquirer, the last one is (theoretically) not.

Data acquisition usually consists in retrieving one or several file(s) on a server (FTP, FTPS, SFTP… you name it). But data can also be provided by other means, such as an attachment (or worse, in the body of an email) or by contacting REST APIs. Once the data is retrieved, it can be normalized by a parser. This part is totally specific to the format provided by the acquirer and requires most of the analysis time in the onboarding process of a new acquirer.

Finally, the reconciliation tries to explain the entirety of each bank movement (wire transfer), by linking all the HiPay payment transactions included in it. The process relies on several matching rules, using transaction references, transaction types, transaction amounts, bank account numbers, banking transaction amounts…

All those parts are asynchronous. Especially for the reconciliation: it is based on three daemons that can be called independently. For this reason (and other implementation choices), it is very complicated to know if one file is completely integrated or not.

Technically, the reconciliation software is based on multiple apps developed in PHP. Those apps use a PostgreSQL database, which content is made available for other software in the company. The asynchronous messages are transported using PGQ and RabbitMQ.

Limitations and future

Due to many different reasons, the reconciliation at Hipay has reached a sticking point. For instance, the technical architecture is not scalable, mostly because it relies on one big database. The software also suffers from a big technical debt, due to years of multiple adaptations by different teams. In fact, the first integration software has been redesigned years ago, to correct some flaws, but the acquirers of the first version have never been transferred to the new one. So today, two versions have to be maintained.

Moreover, with the actual design, it is difficult to extract monitoring information, build KPIs or implement alerting tools from the treated data.

For all these reasons, we decided to migrate our software to Google Cloud Platform. With an entirely new design, we want to make a scalable software with feature deployment, logs for analytics, modern software language and better tooling. Those points will be detailed in other articles.

Give us a little 👏 if you found this to be useful ! or leave a comment if you want more focus put on specific aspects in the future 😉

Thanks for reading !