Setting Up Payments with Express + MongoDB + Square: Considerations & Code

Photo by Blake Wisz on Unsplash

Building out a service to handle user payments requires many considerations. Issues of server (internal and external) failures, double spending, race conditions, database consistency, and durability often arise and are difficult to debug. On top of dealing with software issues, developers have to consider legal issues such as ensuring compliance with Government regulations (ie. PCI). Luckily, there are many software and services that can help alleviate those issues. As an example, by using Square’s payment platform, developers can circumvent the issue of PCI-compliance by not storing and transporting client’s payment information, instead offloading this responsibility onto Square. Check out how to integrate payments with your own website here.

In this post, we will create a simple, durable, and consistent payment service using NodeJs, ExpressJs, MongoDB, and Square. We will be creating an expensive version of Medium, where writers have to pay $1000 for each article publication. Absurd? I think so too. Each article will have a simple title, author, and body fields. I will go through some of my considerations through the process. If you just want to see the code, check it out here.

Note: I welcome any new ideas and corrections. Please leave a comment and I will be sure to read it and correct my article accordingly.

Table of Contents:

1. Schema code
2. Internal and External States
3. Locks in MongoDB
4. Atomicity in MongoDB
5. Sessions and transactions in MongoDB
6. Consistency and durability
7. Conclusion

We will be using Node.js v10.17.+, Express.js v4.17.*, MongoDB v4.2+ with WiredTiger storage engine, Mongoose v4.17.*, and Square to process payments.

Schemas

Let’s start with our schemas. We will have two schemas: one for published articles and one for keeping track of payment states as we make an external request to Square to process payments.

We are using mongoose as library wrapper around MongoDB to help with things like schema validation. The autoCreate flag is used to automatically create a collection if it does not exist already. The actionData in payment.model.js will hold any data that is specific to an actionType (ie. ArticleCreation will need an Article as its actionData).

Internal and External States

When processing payments using an external service like Square, the internal state (our model) and the external state (square’s model) must be synchronized. With how many issues that can arise during its cycle (the payment processing server breaks during request, or internal server breaks while processing response from payment processing server), the payment process needs to be consistent. The way this is achieved is by implementing multi-phase commits. Multi-phase commits are where we step through the process in phases, rollback changes on error, or completely and consistently alter the state of the models on success. For our process, we will be using the following model:

In our prepare phase, we will make sure an identical payment (same author and title) has not been started (pending or done). During our process phase, we will make a external Square request to process the user’s payment. On success, we will reach our update model phase where we create an article and update the payment status in a multi-step transaction to ensure consistency. Once finished, the status will be set to done. If at any point an error is encountered, the payment state will be set to ‘error’, the process aborts, and the database reverts back to an earlier state. In the code, we keep track of the payment states in the payment model’s status property.

Locks in MongoDB

A problem that is as old as time in computing systems is the issue of race conditions. Race conditions occur when two or more processes or threads try to update a shared resource. A very common operation is the test and set, where a condition is checked before something is updated.

Processes often switch between multiple contexts (processes/threads) in order to offer multi-programming, which optimizes process execution time and CPU utilization. The issue arises when two threads of execution try to execute the same operation simultaneously. Lets say the first thread is being processed. First, it checks the test condition to make sure the model is not updated. It passes the test condition, but the process scheduler then switches to the second thread, which also passes the test. The second thread then goes on to update the model which invalidates the test the first thread did. However, the first thread has already passed its check and goes on to update the model as well. One solution would be to make a CPU instruction that does the test-and-set for us, but this is extremely impractical and inflexible. A more common and practical solution involves the idea of mutual exclusion locks.

Locks allow us to create sections of code that are protected, and where only one thread can have access to that section at a time. This is know as mutual exclusion. By using locks, we can lock the section of code that is performing and test-and-set operations so that only one thread can execute it. Once it is done, it can release the lock for other threads.

The WiredTiger storage engine that ships with MongoDB 4.2+ offers document-level concurrency control. It can also detect write conflicts and retry operations. This allows us to use operators like findOneAndUpdate to test-and-set a document atomically and concurrently. This is important in preventing double spending by checking to see if a payment with the same title and author has been initiated already (pending or finished).

In the code below, we take advantage of the concurrency control/document level locking by using a single query to check and create a payment object if it does not exist already (line 20–28).

Two phase commits can also be used as opposed to aggregating all test and set operators under one query. This is done by utilizing a property on the document to keep track of its revision number. Checking that the revision number has not changed and updating the document can be done in a single query, thus preserving concurrency control.

Atomicity in MongoDB

In computer systems, atomicity is the idea that an action (or set of actions) is indivisible, and thus the state of a system while processing an atomic action must either be in the same state before the atomic action or after, never in-between. In MongoDB, single document queries are atomic.

What about when we want to update multiple documents atomically? In our case, we don’t want to set the payment status to ‘Done’ if we did not create the Article document. These two operations must both happen or both not happen. This idea is known as a transaction. MongoDB offers a solution for multi-document atomicity with the use of sessions and transactions.

The code snippet above (‘square_payment.js’, line 191–200, duplicated below) shows how to create a multi-document transaction.

let session = await this.paymentModel.db.startSession();
let paymentUpdate, modelUpdate;
await session.withTransaction(async () => {
        paymentObject.paymentId = squarePayment.id;
        paymentObject.status = 'done';
        paymentUpdate = await paymentObject.save({session: session});
        console.log(paymentObject);
        modelUpdate = await modelUpdateFunction(paymentObject.actionData, session);
        return await Promise.all([
            paymentUpdate,
            modelUpdate,
        ]);
    });
await session.endSession();

In line 191, we create a session using startSession. We then use the withTransaction function to wrap our transaction logic. withTransaction is simply a convenience wrapper that will create the transaction, commit the transaction, and/or abort the transaction for us. Keep in mind that we must pass in the session object to each MongoDB query in order to associate that action with the session, as the changes are only in that session. Once done, we call endSession in line 200 to end the session.

Consistency and durability

So we have mechanisms offered by MongoDB to keep our system in a consistent state in the form of transactions, sessions, and document-level lock supports. However, what happens when we interface with an external payment system? How do we manage and fix and inconsistencies between the two services to make our overall payment flow durable? Consistency is done by using multi-phase commits, and durability can be ensured by using CRON jobs to validate/invalidate payments.

When making payments to Square, we first create our own representation of the payment and set the payment status to ‘pending’. We then make a request to Square to process the payment. Once a response comes back, we update the status to ‘error’ if it fails, or move on to our transaction, in which we create the new Article and set the payment status to done atomically. If at any points in the phase our server or Square’s server fails, we are actually able to bring our system back to a consistent state by using CRON jobs.

// Start CRON job
const msInterval = 1000 * 60 * 5; // run every 5 minutes
setInterval(async () => {
    console.log('Starting CRON job');
    return paymentService.startCronJob()
        .then(res => {
            if ( res ) 
                console.log(res);
        })
        .catch(error => {
            console.log('Error with CRON job: ', error);
        })
        .finally(() => {
            console.log('Finished CRON job');
        })

The startCronJob is from the PaymentServices code above (‘square_payments.js’).

In our CRON job, we first determine any ‘pending’ payments in our database. We get the date range of those payments and make a request to Square using Square’s square-connect package. By matching the square payment’s reference_id with our local model’s _id, we determine which square payment is associated with each article payment, and thus determine if an article has been paid for appropriately. We complete the process by setting the status to error (if there are no matches), or creating an article and setting the status to done (if there is a match).

Conclusion

In this post, we talked about the importance of consistency, atomicity, and durability in a payment system. Consistency is key to keeping a system in a valid state. Atomicity is a key part of consistency which allows us to ensure that a set of operations do not end up in a partial state, either all or none occur successfully. In MongoDB, single document queries are atomic. Multi-document queries require the use of MongoDB sessions and transactions. To keep our system durable, we use CRON jobs to synchronize the external payment system (Square’s services) and our internal model (payment models).

All of my code can be found on Github. This is my first post and I will be sure to improve in the future. Please leave any comments, suggestions, and corrections in the comments. I will be sure to read them.