Realtime Payments — New Payment Platform

By Chris Clark, Engineer at Macquarie Group

I was given the opportunity of onboarding the New Payments Platform (NPP) into Macquarie. NPP enables transactions to be settled from one bank to another in real-time, normally seconds. When I joined the team, it was a blank canvas, there wasn’t a single line of code, the git repos didn’t exist, we didn’t even have build agents. There have been significant changes within my division of the bank, Banking and Financial Services (BFS), Macquarie’s retail banking business, over the past 3 years — we no longer want to run applications on-premises, we run projects using scaled agile, we want to work like tech companies.

Our key principles were:

As a team we added the following to our mantra:

How did we get to the cloud?

Macquarie have invested in ensuring we have access to AWS services for many years and BFS took that and deployed Openshift (K8s platform) on top of it. As a team we didn’t have to use Openshift and we could have deployed our platform on raw AWS if we wanted to. By using the Openshift platform and the tools BFS had built around it, we were able to fulfill a number of our devops objectives, such as centralised logging, continuous deployment, seamless integration with a chat bot for alerting and live performance monitoring of applications.

We had somewhere to run the applications but we also needed somewhere to store the transactions. NPP generates several events per transaction as its processed through the various stages of clearing or aborting, all of which we capture, processed and persist. We decided to use MongoDB Atlas. At first I was skeptical about moving to a schemaless database. I was however sold on the idea we’d be moving to a cloud hosted database and it would be managed by MongoDB.

At Macquarie we have access to other cloud based document datastores but I didn’t want to tie us down to a cloud provider as we’d eventually be running a multi-cloud solution. Using MongoDB Atlas has allowed the team to focus on building the application and less time trying to model the perfect database schema. Upgrades are achieved by rolling the master and upgrading each node one by one whilst the applications and drivers take care of the connection resulting in no outage. We’ve been using MongoDB Atlas for 2 years for Macquarie’s NPP solution, and to date it’s been the easiest part of the solution, it just works.

Thinking Agile

There are two clear distinctive functions of NPP, sending and receiving and we tackled receive first. As this was a new platform with all new applications we didn’t wait for the product to be “ready”. We started to deploy applications daily, getting used to deployments, monitoring, alerting and ensuring that they behaved as expected. We also agreed as a team that we didn’t know everything and what we were building would change, applications would have to change and we accepted that as being agile that was a good thing.

Once receive was proved, we ran a pilot to prove the solution, tuning it, deploying fixes whenever they were required. As a team we don’t wait for the end of a sprint to release, we don’t bundle releases because it’s less effort, we deploy as soon as function is ready. Once receive was done and dusted we started on send which again we tackled incrementally, the first part of the solution was for internal Macquarie accounts. By building out the internal transfers we were able to prove out the large bulk of platform. Again this gave us a chance to learn and adapt whilst also building out the final solution for external transfers. The send part of the solution is 100% separate from the receive allowing us to run the platforms at their own pace.

How Successful is it?

The receive part of the solution went live in June 2019 and the send part of the solution went live in April 2020. We also built the NPP solution with a 2 pizza team, 12 people for those who know.

Once send was released publicly, it simply worked, it was the most anti-climactic release I’ve been involved in but awesome at the same time! We’ve had our moments as a team but we have stuck to our mantra, we are very quick to react, builds are automated, testing is automated and releasing is automated all done within a team of four engineers.

One of the responsibilities of being a participant in NPP is that we must ensure that we are compliant with all of the regulations and procedural guidelines set by NPP Australia (NPPA). We must carry out a full certification process of the platform twice a year. Because we used industry best practice for automation it’s enabled us to carry out the process quickly:

Our track record includes:

Technology trends have changed and so has the way we work. The teams manage their own releases, we decide on what alerts we want to get, we decide on how we are going to build our solutions, we have the correct access to support our applications, we really are empowered to drive the products forward. Oh and one thing I truly love is I’m writing this article on my MacBook pro which to most people will be “so what” or “urghh” but when working for a financial institution it’s gold.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Macquarie Engineering Blog

Sharing insights, innovative ideas and ways of working from Macquarie’s engineering team.