Utilizing Square’s Delayed Capture: Node.js, Express, MongoDB

Photo by Blake Wisz on Unsplash

Square is a payment processing service that allows developers the ease of handling payments. Square’s delayed capture is a mechanism that allows you to hook into the states and events of Square’s payment transaction, allowing developers the ability to integrate their model with Square’s services.

In this post, we will be building an exclusive version of Medium where users will be able to publish an article after paying $1000. With this example, we will explore how to utilize Square’s delayed capture with an Express, MongoDB, and NodeJs backend.

** Update: I realized since my last post that Square’s delayed capture offers better control over the payment process. The current post explains Square’s delayed capture and offers insight into the updated code. Previous post…

Bootstrapping the Project:

In this project, we will be using node v10.17.0 and MongoDB v4.2+ with WiredTiger storage engine. Our package.json will contain the following:

{
  "name": "express-square-node",
  "version": "1.0.0",
  "description": "Simple payments infrastructure scaffolding.",
  "main": "index.js",
  "scripts": {
    "start": "node server.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Eric Ngo",
  "license": "ISC",
  "dependencies": {
    "body-parser": "^1.19.0",
    "crypto": "^1.0.1",
    "express": "^4.17.1",
    "mongoose": "^5.10.5",
    "square-connect": "^4.20200826.3"
  }
}

express will be our backend web application framework. We will be utilizing mongoose as our ODM library to provide schema validation for an otherwise schema-less database. body-parser will provide express middlewares to handle request bodies. crypto and square-connect will be used to interface with Square’s APIs.

Our file structure will be the following:

express-square-node/
├── lib
│   ├── articles
│   │   ├── article.model.js
│   │   └── article.service.js
│   └── payments
│       ├── payment.model.js
│       └── payment.service.js
├── package.json
├── package-lock.json
└── server.js

With the dependencies and project structure out of the way, we can now dive into the business logic.

Articles:

In our application, we need to be able to store published Articles. Thus an important collection we need to create is the Article collection. An Article contains the following as defined by our schema:

/**
 * Article hold published articles. 
 */
const mongoose = require('mongoose');
const ArticleSchema = new mongoose.Schema({
    title: {
        type: String,
        required: true,
    },
    author: {
        type: String,
        default: 'Anonymous'
    },
    body: {
        type: String,
        required: true,
    },
    created: { type: Date, default: Date.now },
}, {autoCreate: true});
const ArticleModel = mongoose.model('Article', ArticleSchema);
module.exports = {
    ArticleModel
};

Using mongoose.Schema allows us to enforce schema validations such as requiring a title to be included in every document of the Article collection. The {autoCreate: true} option tells mongoose to create the collection if it does not exist already. We then create a ArticleModel using the ArticleSchema. Models provide an ODM with which we can interface with our underlying MongoDB database. Don’t be elated yet, we are still going to be using low-level mongodb constructs such as aggregates.

Paired with our ArticleModel is a set of convenience functions that can be shared with other components of our application, which will be our ArticleService. We need to be able to get all articles and create an article.

class ArticleService {
    
    constructor() {
        this.articleModel = ArticleModel;
    }
    /**
     * Returns list of articles
     */
    async getArticles() {
        return await this.articleModel.find({});
    }
    /**
     * Create a new article
     * @param {*} articleData
     * @param {*} session
     */
    async createArticle(articleData, session) {
        let article = new this.articleModel(articleData);
        return await article.save({session: session});
    }
}

getArticles performs a query on the Article collections for all objects. createArtcle creates a new article object and saves the document in a session.

Payments Flow:

Handling payments requires a lot of care, especially since there are legal ramifications/responsibilities when dealing with payments. Fortunately, a lot of issues such as conforming to PCI regulations can be offloaded to payment services like Square. Still, we need to ensure a consistent and logical payments flow for our users. This requires us to synchronize our application model with Square’s transaction states.

Delayed Capture:

Because we want to hook into Square’s payment flow to synchronize the states of our model and the state of the payment transaction, we can use Square’s delayed capture. Delayed capture allows us to hook into the event lifecycle of a transaction and determine the state of the transaction. Square payments have the following states:

• APPROVED: Nonce is verified and can be charged.
• FAILED: Nonce has been declined and can not be charged.
• COMPLETED: Nonce has been charged. End of transaction.
• CANCELED: Nonce has not been charged. End of transaction.

Consistency:

When making payments with Square, we need to synchronize the state of the payment with the state of our application model. We want to make sure that each phase of our payments processed is saved in steps. This can be done with multi-phase commits. We will be using Square’s createPayment (with autoComplete: false), completePayment, and cancelPayment APIs. The phases of our payments flow are as described:

Payment Flow

An article publish request consists of a nonce (generated by the Square’s payment forms library on the client-side) and the user’s article information. When a user makes a request to start a payment (request to create Article), we first check whether a similar payment (similar Article author and title) is currently in a processing state (pending, approved, confirmed) or processed (completed) state. This reduces the possibility of accidental double spending and helps with concurrency. If this is an entirely new and unique payment, we create a payment object to keep track of the payment status as we move along. We then utilize createPayment with delayed capture to check that the nonce is valid. We store Square’s payment response object ID in our paymentObject.paymentId. If the nonce is not approved, we simply set our paymentObject.status to error and end the payment. If the nonce is approved, we set our paymentObject.status to ‘approved’ and create a new article. If there is an error creating the article, we call Square’s cancelPayment and set the paymentObject.status to ‘error’. If successful, we update our paymentStatus to ‘confirmed’. At this point, our model was created successfully and Square has approved the nonce to be charged. There are no logistical complications that would prohibit us from charging the user or require us to cancel the payment. In the confirmed state, we would make a request to Square to completePayment. Once successful, we then set our paymentObject.status to ‘completed’ to complete the payment. This synchronization of payment states allows us to ensure both the application model as well as the payment transaction are in a consistent, finished state.

With an understanding of the overall payment flow, let’s move on to our payment model:

/**
 * Payments hold any pending/processed payments from square.
 */
const mongoose = require('mongoose');
const paymentStatusOptions = ['pending', 'approved', 'confirmed', 'completed', 'error'];
const actionTypeOptions = ['ArticleCreation'];
const PaymentSchema = new mongoose.Schema({
    status: {
        // Current status of payment
        type: String,
        enum: paymentStatusOptions,
        required: true,
    },
    actionType: {
        // Type of action
        type: String,
        enum: actionTypeOptions,
        required: true,
    },
    actionData: {
        // Data associated with action
        type: Object,
    },
    paymentId: {
        // Id from external payment references (in this case from Payment Id from Square)
        type: String,
        default: '',
    },
    created: { type: Date, default: Date.now, required: true },
}, {autoCreate: true});
const PaymentModel = mongoose.model('Payment', PaymentSchema);
module.exports = {
    PaymentModel
};

actionType is used to determine which update function to call for a given model if we want to enforce other payment types. status is important in synchronizing the state of the external payment service and our model update service as described above.

After our payment model, we need to create a payment service to hold all of the payment logic. The payment.service.js code is quite large, so I will only explain certain key functions and patterns. If just want to see the code, the Github link to the project will be at the end of the post.

The nonce and article data is passed to the createArticle function that determines the search query (to ensure unique title and author articles) and setQuery (how to update our model after payments are approved). This information is passed to the paymentsFactory handles the payment flow as described above. Line 33–49 handles creating a paymentObject IF a similar paymentObject is not being processed or has been processed. It does so by using the $setOnInsert operator that only updates the model if it is new. I take advantage of the new: false to determine whether or not we have created a new document. With new: false set, findOneAndUpdate will return an old revision of an object that matches the searchQuery, which IF null means that a new object has been created through that query. We then approve the payment using thesquare-connect library. Once approved, we can create an article. In line 80–96, we atomically create an article by calling the article’s modelUpdateFunction and set the paymentObject’s status to confirmed. If there is an error in processing, we simply cancelApprovedPayment and end execution flow. Again, the full code will be in the Github down below.

Handling Concurrency:

There are many points of the program in which concurrent execution may be an issue. This is apparent in line 33–49 where there seems to be a race condition. We are testing some condition, then setting some value. We can handle this with pessimistic concurrency control, whereby we employ mutex locks to protect that critical section. Fortunately, this can also be handled by the storage engine. WiredTiger offers document-level concurrency with optimistic concurrency control. With OCC, the engine can optimize queries by grouping queries that can share locks. This allows us to use operators like findOneAndUpdate to test-and-set a document atomically and concurrently.

Multi-Document Atomicity:

We know that document updates are atomic. But what happens when we want to update two documents at once? In our case, we don’t want to set the payment status to ‘confirmed’ if we did not create the Article document. These two operations must both happen or both not happen. This idea is known as a transaction. MongoDB offers a solution for multi-document atomicity with the use of sessions and transactions. With transactions, we start by starting a transaction in a session. With the session, we would make any updates we would like. It is important to note that at this point, only the current session can see any updates performed in a transaction before the transaction si committed. Thus it is important to pass the session to other functions/components that may perform mongoDB queries as part of the transaction. Once all updates have been made, the transaction can be committed and ended. If there are any errors, the transaction can simply be rolled back. Transactions offer multi-document atomicity by ensuring that all the updates are committed or rolled back at once, ensuring that all operations happen or none of them happen. Line 82–91 of make_payment.js has an example of utilizing transactions. In this case, session has a handy withTransaction method that will perform all operations inside a transaction.

Making The System More Durable:

Durability is a part of ACID, which is a term used to describe the properties of a database. Durability refers to the property of persisting data through software or hardware failures. What this means is that if a payment is currently being processed and a server outage occurs, once our server starts back up, we want to be able to put the payment in a finished state (completed or errored). We can make our system more durable and self-healing by adding a CRON job whose job is to finish up any payments in an incomplete state.

First, we determine a cutoff time so that we do not consider payments that have been newly created. Lines 8–19 search for any payments that are before the cutoff time and are in an unfinished state (pending, approved, confirmed). Lines 25–43 determine how to finish each of the payments.

Conclusion:

Square’s delayed capture offers developers the ability to hook into the lifecycle of a transaction, allowing the ability to synchronize application models with transaction states. Concurrency is handled by the WiredTiger storage engine which offers optimistic concurrency control for both test-and-set protection and optimization through shared locks. Consistency is handled using multi-phase commits and synchronization of the application model and square payment model via delayed capture. Durability is handled using CRON jobs that fix up any payments in an incomplete state. This post is an update and supplement to my previous post.

Further:

• Build out the Frontend: Reference Square’s documentation to learn how to utilize SqPaymentForm to generate Nonces on the client-side here.
• Set up HTTPS and generate SSL certs to ensure encryption of messages. There are a lot of tutorials that cover this process.
• Enforce document uniqueness by using unique indexes on the document model itself as opposed to utilizing a secondary model like Payment.

Code:

The working backend of the application can be found here.