CQRS From Scratch With TypeScript

Implementing basic CQRS pattern using Node.js and TypeScript

Photo by Caspar Camille Rubin on Unsplash

About

I’ve been wanting to try out the CQRS pattern for some time now, ever since my best friend Tony started telling me about it.

I finally had some free time yesterday and took a few hours to throw together enough of a project to play around with the main concept behind CQRS.

So, what exactly does CQRS stand for?

According Martin Fowler, who is well known in the software industry:

CQRS stands for Command Query Responsibility Segregation. It’s a pattern that I first heard described by Greg Young. At its heart is the notion that you can use a different model to update information than the model you use to read information.

Basically what this means is that we can keep all of our write logic separate from our read logic.

This separation allows us to optimize the two pieces individually.

For example, if we are sending a query we can bypass the application AND domain layers, sending the query straight to the infrastructure level.

While CQRS is not normally needed for most applications, it is really nice for larger projects that will need to scale and grow. A lot of CQRS applications use events and event handlers, making it even more scalable because you can use a bunch of services that can communicate with each other through events.

CQRS is not for every app, and adds a lot of complexity up front. If you are working on an app that would benefit from splitting up your commands and queries, then its definitely an interesting idea to check out.

According to Martin Fowler, CQRS would be optimal for you project if performance is an issue, or you have have way more reads than writes:

CQRS allows you to separate the load from reads and writes allowing you to scale each independently. If your application sees a big disparity between reads and writes this is very handy. Even without that, you can apply different optimization strategies to the two sides. An example of this is using different database access techniques for read and update.

Check out the rest of his article here.

Setting Up

(Full code is at this github)

Now that we know a little more about what CQRS entails, we can go ahead and start building the project.

We’re going to be using TypeScript with Nodes.js for our server, so go ahead and create a new project directory, and initialize it with npm.

Fill out the information when prompted (or add ‘-y’ after npm init to accept all defaults) and then cd into the new directory:

$ mkdir cqrs
$ cd cqrs
$ npm init

Let’s install some TypeScript now:

$ npm install -D typescript

This will add typescript to our package.json dependencies.

Next step is to create our TypeScript config file. Make a new file in our project folder called ‘tsconfig.json’ and put the following in it:

For brevity I won’t go into the tsconfig.json file, but you should definitely check out the documentation here if you are unfamiliar with tsconfig.json.

Next step for setting up our development environment is to install nodemon from npm, which will automatically run the script to start the server every time we make a change and save it in our source code.

It may sound tiny but it’s actually a huge deal, and developing backend apps with node.js can be very frustrating without it.

To use nodemon with TypeScript we’re going to download ts-node, which will compile our TypeScript code into JS which nodemon can then run with Node.js to start our server.

$ npm install -D nodemon ts-node

Now we need a couple scripts in our package.json:

The “start” command will start the server using ts-node with a debugger attached, and the “start:watch” command will run nodemon.

In order for nodemon to know enough information about our environment we have to create a “nodemon.json” file.

You could also add the nodemon config to your package.json file but it tends to get cluttered most of the time so I would prefer to keep it clean.

Create a new file called “nodemon.json” and fill in some basic information. A good resource for configuring nodemon can be read here.

Now when nodemon runs it will know which files to watch for changes, as well as the command to run when the restart is triggered.

Let’s take it for a quick spin. Make a new folder ‘src’, and inside ‘src’ create ‘application’. Inside ‘application’ folder add a “server.ts” file.

Type something like:

console.log("Hello World!")

Now save it and from the command line type “npm run start:watch”.

You should see something similar:

Now change “Hello World!” to “Goodbye World!” and save it. The server will automatically reload with the updated source and you will see the updated output.

Now that we have that out of the way we can lay out the hierarchy we’ll be using for our app.

Project Architecture

For our project, we’re going to be using an architecture that can immediately realize the benefits of the CQRS pattern.

Commands

We start with the presentation layer, which communicates directly with the client. In this case through HTTP requests received by our express.js server.

After the presentation layer receives a message, we will call into the application layer which will deal with things such as validation, security, or authentication.

The application layer will then call into the domain layer, which is in charge of all the logic that is related to specific domains. For our little app the only entity we have implemented is the Game (just tossed in a previous projects sqlite database), so our only domain right now is Games.

Now this is where this pattern really shines, because we can now easily add more bounded contexts within our domain layer. We can add an ‘Orders’ domain, or a ‘Users’ domain, in a modular and clean way.

This makes it easy to keep our code organized easy, since we do not have to modify any logic relating to the other domains. There is a clear separation of concerns.

This domain layer would then call into the infrastructure layer to persist the updated game. The infrastructure layer would contain the logic for persisting or fetching data through various mechanisms such as Redis cache or a Database.

In our case we are using TypeORM, which is based on the Repository pattern.

Queries

For Queries we can skip both the application layer and the domain layer, and send the queries straight to the infrastructure layer.

This is where we begin to see performance boosts, especially when our project is heavy on reads.

Project Directory

Let’s go ahead and create some folders.

In the ‘src’ folder create a folder for ‘commands’ and a folder for ‘queries’. We can immediately begin separating the two elements, which would make it easier for splitting our app into two and hosting each piece on its own box.

In the ‘commands’ folder lets make a folder for the ‘application’ layer, which as a reminder will be dealing with application level logic such as validation and authentication.

Inside the application folder we will put a folder to hold the commands for each entity. In our case we have ‘games’ and ‘users’, but you can already see how easy it would be to scale this indefinitely.

Do the same thing with a ‘domain’ folder inside the ‘commands’ folder, as this will give us clear namespace separation when we’re writing our code.

Go ahead now and add a ‘queries’ folder, and a subfolder for each entity.

Go ahead and also make a directory and name it ‘common’. This is where we will store our interfaces we declare, since they will be used amongst all the various layers and entities.

Create a directory for ‘routes’, as this will basically function as our presentation layer. It is the layer where we receive instructions from the client and is the closest to the user.

Finally, create one last directory named ‘entity’, as this is where we will store our repository (table) definitions that TypeORM will use to interact with the database.

Your project should resemble this:

Routes

It’s been awhile since we’ve been able to write any code, so let’s set up our server entry point and create a route so we can work through the implementation of working with the various architectural layers.

In server.ts setup a barebones express server with a simple hello world route to make sure its working:

Notice we are also using morgan for logging, so you will need to install morgan, express, and the type definitions. Check your terminal to see whats missing and install them. I had to install ‘express’, ‘@types/node’, ‘morgan’.

Now go to the browser and navigate to the address where the server is listening and you should see a ‘Hello World!’. Success!

Now let’s hook up a ‘Games’ route that will serve all of our ‘Games’ related functionality.

Create a file in the ‘routes’ folder named ‘games.ts’:

Import this route back into the ‘server.ts’ file and tell our express server to use these routes for any request sent to ‘/games’.

Go ahead and point your browser to the new ‘/games’ route and make sure your wired up right:

Now we have our routes setup, and you can see how easy it would be to start adding routes for each entity in our project.

Interfaces

Now that we have our project structure setup, let’s create some interfaces that we can use to implement the functionality of our CQRS pattern.

For this project it is advisable to use the command pattern as a way to communicate between the different architectural layers.

Commands

We are going to create command interfaces that will require an execute function where we’ll be putting the logic for the command.

In the common folder makes some files for each interface we will use.

We will use four separate pieces for our command process:

1. The command itself
2. A command config, which will be sent to the command factory with the information required to create a command
3. A command factory which will be used to create a specified command
4. A command response that is returned when the execute function is used on a command, indicating if the command was successful or not.

Here’s the implementations for the different interfaces. Notice the naming indicates with an I that the code represents an interface.

ICommand.ts

ICommandConfig.ts

ICommandFactory.ts

ICommandResponse.ts

Now we have the basic building blocks for creating commands that can send instructions into the different architectural layers.

TypeORM

In order to complete the example workflow for the create game command we need to configure our infrastructure layer, which in this case will be the TypeORM library.

TypeORM exposes our app to a provider that uses the Repository pattern to wrap our entity objects with. This will give us access to some very handy methods for dealing with the data store.

Entity

TypeORM expects us to define an entity to represent the shape of our tables, which it then uses for validation and type checking to ensure the objects we pass into it are the correct objects for the database table.

Let’s create a quick entity to represent our game object we will be creating:

Create a new file in the ‘entity’ folder named ‘Games.ts’. This will represent our games table:

Now that we have this entity we can go ahead and use it as our TypeScript definition as well.

Next let’s wire up our TypeORM provider in our ‘server.ts’ file.

Install TypeORM with npm install --save typeorm and update the ‘server.ts’ file to wrap the express server with the TypeORM provider:

Final step for TypeORM to run is to write the config file. This is not necessary, as you can write this directly into your code, but once again I find it convenient to be able to put a ‘ormconfig.json’ file into the root of the project.

If your using the database from this example, or would like to get this running easily, make sure you install ‘sqlite3’ with npm: npm install --save sqlite3

Create Game Example

Finally, we will create a simple workflow that takes add game command from the client and see how we can implement the CQRS command side of the pattern.

Presentation

The process starts at the presentation layer. This is where the command originates as the result of an action invoked by the user.

The presentation layer in our case is the games route, so lets create a route that indicates a game should be created, and make sure it can handle a game object sent along with the request:

First add a new route to handle ‘POST’ requests that are sent to the ‘/games/create’ route:

We bring in our Application layer command factory that contains the commands we can use to talk to the application layer. You can see here that the presentation layer has no connection with the domain or the infrastructure layer, which is very good for security.

The AppGameFactory:

Basically this is just a switch that takes the config object and returns the instantiated command. Using this pattern makes it easy to keep all the commands visible and in one place. If we ever want to switch out a version of a command, for example update the ‘POSTCommand’ to use another command, it’s as easy as changing the factory.

Let’s implement the ‘POSTCommand’, which is in charge of sending information down into the domain level from the infrastructure level.

We call it POSTCommand because it’s created as a result of a ‘POST’ HTTP request.

This is a simple command at the application layer. This is where you would deal with validation and authentication, but in our case we are keeping it simple to understand the flow better.

We instantiate a copy of the DomainGameCommands, which only deals with the logic needed to manipulate games in the domain layer. Once again this keeps our logic separate and helps keep everything organized and scalable.

We use the CRUD model when dealing with this layer as opposed to HTTP verbs, since we’re closer to the data store and it makes more sense to use verbs associated with data persistence.

Once again, the CRUDCommandFactory is just a switch/if-else statement that returns the correct command instance for the verb we send in.

Now that we have a factory that can give us our ‘CreateGame’ command, let’s go ahead and look at the implementation for the command at the domain level. Remember that this is where put our domain specific logic, and is the only layer that can call into the infrastructure layer for the command side of our application.

This is the final command here. We instantiate a ‘Games’ entity and transfer the values from our Game object that we’ve been passing along.

Then we use the ‘getManager()’ function from TypeORM to grab a reference to our ‘Games’ repository. This will allow us to use the methods from the repository to do things such as add and update the database, without having to implement the logic ourselves.

Now that we have a reference to our ‘Games’ repository, we call save and then either return ‘true’ or ‘false’ depending on the success of the ‘.save’ call.

In this case TypeORM is our infrastructure level, and we call into it directly from the domain layer, completing our one way trip through the command process.

Testing

Go ahead and use postman to send a post of a valid game object:

{"Name": "sample", "RawgID": 20, "Metacritic": 100, "Genres": "Action", "Indie": false, "Presence": 83, "Platform": "Windows", "Graphics": "4gb GPU", "Storage": "180gb", "Memory": "8gb", "RatingsBreakdown": "34/45/15", "ReleaseDate": "January 14, 2020", "Soundtrack": false, "Franchise": true, "OriginalCost": "$39.99", "DiscountedCost": "$39.99", "Players": "singleplayer, multiplayer", "Controller": true, "Languages": "English, Mandarin", "ESRB": "Teen", "Achievements": 55, "Publisher": "idSoftware", "Description": "lots of stuff", "Tags": "Fun, Violent", "SteamURL": "https://store.steampowered.com/app/42700/?snr=1_5_9__205"}

And you will see the server spit out a success message indicating that the game was added to the database, and you will see a ‘200’ status code in whatever your calling the server from (postman in my case) indicating success:

Next Steps

That is a lot of information for one article, so I will create another one that outlines the flow of the queries. The query side is much simpler as it goes straight from the presentation layer (routes) to the infrastructure layer (TypeORM).

Conclusion

I know this was a lot of information, and I hope I was able to get it all down without making too many errors. Please let me know if you find an error or if there is a concept that is not explained clearly, so I can rethink the way I presented it.

This was my first attempt at implementing the CQRS pattern, and even though it is pretty basic it is still possible to see how powerful this pattern could be, especially in very large applications which need to be very performant or have very high number of reads to writes.

Let me know if you have any advice or ways to make it better, I always appreciate the opportunity to discuss these ideas with anyone else!

The full code is hosted on github at this url.