Advanced Transaction Management with NestJS & TypeORM
IMPORTANT NOTE:
If you want to follow along easier in the advanced transaction management part, clone the following repository: https://github.com/muhammetozendev/nestjs-transaction-management
Introduction — Understanding Transactions
In this article I’ll be talking about how to deal with transactions in NestJS. Before we begin, it’s crucial to understand what a transaction is (if you know what is a transaction, you can skip to the second section). Let’s imagine a scenario where user tries to place an order. Assume that there are three database tables: products, orders and items. Obviously, the products table contains all the products in our app. The orders table stores information about the order like order number, date, and lastly, the items table stores information about which items/products were bought in a specific order along with their quantity. Consider the following tables:
As we can see, we have two products and the other two tables (orders & items) are empty as we have not places any order just yet. If we are placing an order by sending a request to an endpoint like POST /api/place-order this endpoint would send 2 write queries to the database. One query will insert order information into the orders table to create an order object and the other one will insert the item/product information and that query is essential to store what items were bought in that order.
If both of these queries work properly, we are lucky and here’s what the database will look like:
Here, we have one order object with id 100 and in the orders table, and we have 2 other objects associated with that order in items table. The first one is computer (because product id is 1) and the quantity for computer is also 1. The second object is mouse and quantity is 3. Everything looks good so far. But what if the first write query succeeds so we can create the order object but the second query to insert the items fails. In that case, the API call will fail and an error response will be sent back to the user but then we will have inconsistent data in our database because order object will be left in orders table without any items. Here’s how database will look like when one query succeeds and the other one fails:
In such a case, there should be no order object whatsoever as the API call failed. So this failed API call should not mess with any table or record. And as we can see up there, we have an order object but no items associated with it. That’s actually where transactions come in. As soon as you create a transaction, nothing will be written permanently into the database unless you perform COMMIT. If you do not COMMIT the transaction, it will be as if you never executed those queries. We only commit once we are sure all the write queries we are executing succeeds. If something goes wrong and you want to revert all the modifications you have performed in that API call (insert, update, delete queries), then you want to perform a ROLLBACK which reverts any modification that was made since the beginning of the transaction.
As you might have guessed, we will start a transaction in POST /api/place-order endpoint and attempt to execute those two write queries within the transaction. If we get no exceptions, we commit the transaction which persists all the writes into the DB. If any exception was thrown, we will rollback so that all the write operations are reverted and that failed API call will not modify anything in the DB. So the rule is any time you have to execute multiple different write queries in a single route, you have to wrap them inside a transaction. Now that we have a good understanding of transactions, we can continue on.
Different Ways of Implementing Transactions
In this article, I’ll be showing you how to implement transactions in two different ways. In the first way, we’ll be simply using query runner object to start a transaction. Then all the write queries are going to be run using that query runner object to ensure we wrap all the write operations in that transaction. The second way will be the advanced one where we will utilize interceptors and request scoped repositories to automatically wrap all the queries of a specific route in a transaction so we don’t have to create the query runner object manually each time.
NOTE:
I just want to note that this database & API we are building is not designed for production use as we are missing a lot of concepts like authentication, user registration etc. I’ve created this app only to be able to implement and explain transactions easier.
Simple Implementation
As I stated above, this is a simple approach where we create a query runner, get a connection, start a transaction, and execute all queries over that query runner. Imagine we have two TypeORM entities as Order and Item . The way we would execute queries without transaction would be as follows:
constructor(
// first inject repositories
@InjectRepository(Order)
private orderRepository: Repository<Order>,
@InjectRepository(Item)
private itemRepository: Repository<Item>
) {}
@Post()
async routeHandler() {
// then make queries
await this.orderRepository.save({
/* order data */
});
await this.itemRepository.save({
/* Item data */
});
// ...
}However, as we mentioned before, this is a bad approach and we need to wrap these two queries in a transaction. Here’s how to implement transactions using query runner object:
constructor(
// inject data source
private dataSource: DataSource
) {}
@Post()
async routeHandler() {
const queryRunner = this.dataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();
try {
await queryRunner.manager.save(Order, {
/* order data */
});
await queryRunner.manager.save(Item, {
/* item data */
});
await queryRunner.commitTransaction();
} catch (e) {
await queryRunner.rollbackTransaction();
throw e;
} finally {
await queryRunner.release();
}
// ...
}Here there are a couple things to point out:
queryRunner.connect()method reserves a connection from the connection pool.queryRunner.startTransaction()is an async method that creates the transaction.- We use
queryRunner.managerobject to access the entity manager which allows us to perform database queries over that connection that opened the transaction. That means all write queries we send over this entity manager object will be wrapped inside the transaction. Entity manager exposes methods likesave,find,findOneetc. Also when using entity manager like that, the first argument has to be the entity class so that the entity manager will know what table we are inserting into. - Once we execute all the queries without any errors, we run
queryRunner.commitTransaction()to commit the transaction which persists the writes into the DB. - If any error occurred, we run
queryRunner.rollbackTransaction()to revert all the changes and throw the error to be handled by NestJS - And lastly, we run
queryRunner.release()method in the finally block to return the reserved connection back to the connection pool. This step is very important and if you don’t release the connections you reserved, that will lead to performance issues.
We just covered a very basic way to run queries in a transaction just to understand how it works. In the remaining sections of the article, I’ll set up a new project from scratch where I’ll be showing an advanced way to manage transactions using interceptors, request-scoped repositories and custom decorators. If you like advanced stuff, stick around until the end 😃
Advanced Transaction Management
Now we know how to implement transactions at a basic level, it’s time to implement transactions using repository design pattern and request-scoped providers.
Note: I assume you already know how to use TypeORM and understand NestJS ecosystem at a basic level. The focus on this project will only be on implementing transactions. Also we will only create a minimal number of endpoints needed to demonstrate how to implement transactions. If you want to add a couple more endpoints to extend the functionality, feel free to do so.
Database Design
Let’s start with DB design. As we can see up there, we have a product table which holds product records. The order table is responsible for keeping order related info and lastly we have the item table which holds the items (products) bought in a specific order.
As for the relations, there’s a one-to-many relationship between order and item tables as one order could have many items. There’s also another one-to-many relationship between item and product because one product could appear in item table multiple times. We will keep those relationships in mind when implementing entity classes.
File Structure
/src
/common
base-repository.ts
transaction.interceptor.ts
/modules
/items
/dto
create-item.dto.ts
items.entity.ts
items.module.ts
items.repository.ts
items.service.ts
/orders
orders.controller.ts
orders.entity.ts
orders.module.ts
orders.repository.ts
orders.service.ts
/products
products.controller.ts
products.entity.ts
products.module.ts
products.repository.ts
products.service.ts
app.module.ts
main.tsAs we can see, we have 3 different modules:
- Items: Responsible for adding items to an order
- Orders: responsible for managing orders
- Products: responsible for retrieving products in database
Common
We will go into more detail about each module but first I want to have a look at the common directory:
// transaction.interceptor.ts
import {
CallHandler,
ExecutionContext,
Injectable,
NestInterceptor,
} from '@nestjs/common';
import { Request } from 'express';
import { Observable, catchError, concatMap, finalize } from 'rxjs';
import { DataSource } from 'typeorm';
export const ENTITY_MANAGER_KEY = 'ENTITY_MANAGER';
@Injectable()
export class TransactionInterceptor implements NestInterceptor {
constructor(private dataSource: DataSource) {}
async intercept(
context: ExecutionContext,
next: CallHandler<any>,
): Promise<Observable<any>> {
// get request object
const req = context.switchToHttp().getRequest<Request>();
// start transaction
const queryRunner = this.dataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();
// attach query manager with transaction to the request
req[ENTITY_MANAGER_KEY] = queryRunner.manager;
return next.handle().pipe(
// concatMap gets called when route handler completes successfully
concatMap(async (data) => {
await queryRunner.commitTransaction();
return data;
}),
// catchError gets called when route handler throws an exception
catchError(async (e) => {
await queryRunner.rollbackTransaction();
throw e;
}),
// always executed, even if catchError method throws an exception
finalize(async () => {
await queryRunner.release();
}),
);
}
}Whenever this interceptor is applied to a route, we start a transaction and attach an entity manager containing the transaction to the request with the key ‘ENTITY_MANAGER’. We export that value in a constant to avoid potential typos.
Next up we run the actual request handler method with next.handle() . The pipe() call right after takes 3 argument. First one will be a type of map or tap function that will handle response data upon successful handling of the request. There we COMMIT the transaction and return the data as it is. The second argument is an error handler and is run when controller throws exception. In that function, we ROLLBACK as the API call simply failed and throw the error. Last argument which is the finalize method will always be executed. We use that function to release the connection we reserved from the connection pool. This step is very important.
Next important class we have in common folder is base repository:
import { Request } from 'express';
import { DataSource, EntityManager, Repository } from 'typeorm';
import { ENTITY_MANAGER_KEY } from './transaction.interceptor';
export class BaseRepository {
constructor(private dataSource: DataSource, private request: Request) {}
protected getRepository<T>(entityCls: new () => T): Repository<T> {
const entityManager: EntityManager =
this.request[ENTITY_MANAGER_KEY] ?? this.dataSource.manager;
return entityManager.getRepository(entityCls);
}
}This class is supposed to be the parent class for all the repositories and it takes two arguments from the constructor:
- dataSource: A simple connection from the connection pool that doesn’t have any transaction
- request: The request object. We have direct access to the request object because we will define our repository classes as request scoped.
Then we can see that there’s a protected method whose only task is to get an entity manager from the request object if present. We can only get the entity manager from request if the above interceptor was used. If there’s no entity manager in the request object, it means the operation is not meant to run in a transaction and we get the entity manager from the dataSource object. Once we get the entity manager, we will call getRepository method by passing in the proper entity class to get the corresponding repository instance.
Simply put, this method will give us a repository instance and that repository will run queries inside a transaction if the transaction interceptor is used. If transaction interceptor was not used, it will run queries outside of any transaction.
Repositories
Next up, we have repository classes. The responsibility of repository classes is to perform any kind of database interactions. The benefit of extracting the database communications to another layer is to have a cleaner design and it also makes unit testing easier. With that said, we need to look into 3 repository classes in to have a better understanding of the project:
// products.repository.ts
import { Inject, Injectable, Scope } from '@nestjs/common';
import { REQUEST } from '@nestjs/core';
import { Request } from 'express';
import { BaseRepository } from 'src/common/base-repository';
import { DataSource } from 'typeorm';
import { Product } from './products.entity';
@Injectable({ scope: Scope.REQUEST })
export class ProductsRepository extends BaseRepository {
constructor(dataSource: DataSource, @Inject(REQUEST) req: Request) {
super(dataSource, req);
}
async getAllProducts() {
return await this.getRepository(Product).find();
}
}// items.repository.ts
import { Inject, Injectable, Scope } from '@nestjs/common';
import { REQUEST } from '@nestjs/core';
import { Request } from 'express';
import { BaseRepository } from 'src/common/base-repository';
import { DataSource } from 'typeorm';
import { Item } from './items.entity';
import { CreateItemDto } from './dtos/create-item.dto';
@Injectable({ scope: Scope.REQUEST })
export class ItemsRepository extends BaseRepository {
constructor(dataSource: DataSource, @Inject(REQUEST) req: Request) {
super(dataSource, req);
}
// Create multiple items
async createItems(orderId: number, data: CreateItemDto[]) {
const items = data.map((e) => {
return {
order: { id: orderId },
product: { id: e.productId },
quantity: e.quantity,
} as Item;
});
await this.getRepository(Item).insert(items);
}
}// orders.repository.ts
import { Inject, Injectable, Scope } from '@nestjs/common';
import { Request } from 'express';
import { BaseRepository } from 'src/common/base-repository';
import { DataSource } from 'typeorm';
import { Order } from './orders.entity';
import { REQUEST } from '@nestjs/core';
@Injectable({ scope: Scope.REQUEST })
export class OrdersRepository extends BaseRepository {
constructor(dataSource: DataSource, @Inject(REQUEST) req: Request) {
super(dataSource, req);
}
async getOrders() {
return await this.getRepository(Order).find({
relations: {
items: {
product: true,
},
},
});
}
async createOrder(orderNo: string) {
const ordersRepository = this.getRepository(Order);
const order = ordersRepository.create({
date: new Date(),
orderNo: orderNo,
});
await ordersRepository.insert(order);
return order;
}
}As we can see, all these repositories extend the base repository and they all call have a super call to pass data source and request objects to the parent class constructor (BaseRepository). Also we see that all of these repositories are request scoped, which means that they are reinitialized on every request which allows Nest to inject the request object so we can know if there’s an active transaction in the request.
In products repository, we only have one method to get all the products. We don’t have all crud operations in products repository because we will create the test data manually.
In items repository, we have only one method to create multiple items that belong to a single order. By using map function, we take the dto and parse it into an array of properly formatted item objects and save them all in the database. And here’s how CreateItemDto looks like:
import { IsNumber } from 'class-validator';
export class CreateItemDto {
@IsNumber()
productId: number;
@IsNumber()
quantity: number;
}Lastly, we have orders repository where we can get all the orders and we can also create an order given the orderNo.
Entities
Now it’s time to talk about entities. Lets remember our database design:
Entities are representation of database tables in TypeORM and we have to define entities in a way that they will reflect the above database tables. Here are the entity classes:
// items.entity.ts
import { Column, Entity, ManyToOne, PrimaryGeneratedColumn } from 'typeorm';
import { Order } from '../orders/orders.entity';
import { Product } from '../products/products.entity';
@Entity()
export class Item {
@PrimaryGeneratedColumn()
id: number;
@Column()
quantity: number;
@ManyToOne((type) => Order, (order) => order.items)
order: Order;
@ManyToOne((type) => Product, (product) => product.items)
product: Product;
}// orders.entity.ts
import { Column, Entity, OneToMany, PrimaryGeneratedColumn } from 'typeorm';
import { Item } from '../items/items.entity';
@Entity()
export class Order {
@PrimaryGeneratedColumn()
id: number;
@Column()
orderNo: string;
@Column({ type: 'datetime' })
date: Date;
@OneToMany((type) => Item, (item) => item.order)
items: Item[];
}// products.entity.ts
import { Column, Entity, OneToMany, PrimaryGeneratedColumn } from 'typeorm';
import { Item } from '../items/items.entity';
@Entity()
export class Product {
@PrimaryGeneratedColumn()
id: number;
@Column()
title: string;
@Column()
price: number;
@OneToMany((type) => Item, (item) => item.product)
items: Item[];
}As we can see, we have defined all the entities by defining required columns in each one and associating them with each other using @ManyToOne and @OneToMany decorators. We have one to many relationship between orders and items and also another one to many relationship between products and items. Now that we understand the entities, we should define other components like controllers and services for each module.
Controllers & Services
Let’s start by defining what is a controller and a service. Controllers should only be responsible for handling HTTP requests and responses. However, services are meant to contain the actual business logic. So we keep the business logic away from the controllers. With that explained, we can now continue explaining every module.
Products Module
// products.controller.ts
@Controller('products')
export class ProductsController {
constructor(private productsService: ProductsService) {}
@Get()
async getAllProducts() {
return await this.productsService.getAllProducts();
}
}// products.service.ts
@Injectable()
export class ProductsService {
constructor(private productsRepository: ProductsRepository) {}
async getAllProducts() {
return await this.productsRepository.getAllProducts();
}
}As we can see from the products controller, we only have one endpoint defined in this module:
GET /products
This endpoint allows us to fetch all the products. We did not implement all the endpoints as we won’t need them for demonstrating transactions. That controller calls the service method getAllProducts which then calls the products repository to get all products.
Items Module
In items module, we do not have a controller as we don’t want to expose any functionality inside of items.service.ts via an HTTP endpoint. Items service will be used by orders service instead. Here’s the implementation for items service:
@Injectable()
export class ItemsService {
constructor(private itemsRepository: ItemsRepository) {}
async createItems(orderId: number, items: CreateItemDto[]) {
await this.itemsRepository.createItems(orderId, items);
}
}And I think it’s pretty self explanatory. It just takes order id and array of items and create them all in the items table calling createItems method from items repository we just discussed above.
Orders Module
// orders.controller.ts
@Controller('orders')
export class OrdersController {
constructor(private ordersService: OrdersService) {}
@Get()
async getOrders() {
return await this.ordersService.getOrders();
}
@Post()
@UseInterceptors(TransactionInterceptor)
async createOrder(
@Body(new ParseArrayPipe({ items: CreateItemDto }))
data: CreateItemDto[],
) {
return await this.ordersService.createOrder(data);
}
}As we can see, orders controller exposes 2 endpoints:
GET /ordersPOST /orders
These endpoints will be used to retrieve the orders as well as creating a new order. Let’s take a look at the service methods being called to understand the business logic:
@Injectable()
export class OrdersService {
constructor(
private ordersRepository: OrdersRepository,
private itemsService: ItemsService,
) {}
async getOrders() {
return await this.ordersRepository.getOrders();
}
async createOrder(items: CreateItemDto[]) {
const orderNo = `ORD_${randomUUID()}`;
const order = await this.ordersRepository.createOrder(orderNo);
await this.itemsService.createItems(order.id, items);
return order;
}
}The getOrders method is pretty simple. It just gets all the orders from database by making a call to getOrders method of ordersRepository. However the interesting part is we are also injecting items service in the constructor. When we look at the createOrder method, we have a list of items to save under a specific order. We first generate a random order id, create the order and then create the items under it. However, creating both order object and items will require at least 2 queries. That’s why we need transaction for this endpoint. If you look carefully at orders controller, we have annotated createOrder route with @UseInterceptors(TransactionInterceptor) . This simply ensures all write operations will be wrapped in a transaction in this specific route and repositories first look at the request and see if there’s any active transactions attached, if there is one, that transaction entity manager is used to wrap everything in a transaction. That way, the 2 queries we send in create order (first: createOrder, second: createItems) will be wrapped in a transaction.
There’s one thing to note here. It is not possible to inject a service from another module unless you import the whole module. For this setup to work, items module must export its service and orders module must import items module to be able to use its service. Here are the module files for both items and orders modules:
@Module({
providers: [ItemsService, ItemsRepository],
exports: [ItemsService], // items module exports the service
})
export class ItemsModule {}@Module({
imports: [ItemsModule], // orders module imports items module
controllers: [OrdersController],
providers: [OrdersService, OrdersRepository],
})
export class OrdersModule {}With exports and imports in place everything will work.
App Module
Last thing I want to mention is app module which is the root module that every nest application has. There I have written a simple logic to create some products at application start so we don’t have to do it manually by writing a sql script.
@Module({
imports: [
ConfigModule.forRoot(),
TypeOrmModule.forRoot({
type: 'mysql',
host: process.env.HOST,
username: process.env.DB_USER,
password: process.env.DB_PASS,
database: process.env.DATABASE,
entities: [Product, Order, Item],
synchronize: true,
logging: true, // log all the queries
dropSchema: true, // start with a clean db on each run, DO NOT USE FOR PRODUCTION
}),
ProductsModule,
OrdersModule,
ItemsModule,
],
controllers: [],
providers: [],
})
export class AppModule implements OnModuleInit {
constructor(private dataSource: DataSource) {}
async onModuleInit() {
await this.dataSource.query(`
insert into product (title, price) values
('Computer', 1000), ('Mouse', 19);
`);
}
}As we can see, we are inserting computer and mouse records to the database whenever application initializes. Also, we can see the database configuration and all the options in TypeOrmModule.forRoot call.
Results
Let’s manually test out the endpoint by sending an http request to see if the queries are actually run inside a transaction. You can use any http client for that purpose.
We send the following request
POST http://localhost/orders
Content-Type: application/json
[
{
"productId": 1,
"quantity": 3
},
{
"productId": 2,
"quantity": 5
}
]And here’s the the response:
{
"orderNo": "ORD_1dd3570d-b82e-4a8d-bb97-8a3b42766302",
"date": "2023-09-17T21:50:06.434Z",
"id": 1
}Also when we look at the console logs, we see the following queries being executed:
START TRANSACTION
INSERT INTO `order`(`id`, `orderNo`, `date`) VALUES (DEFAULT, ?, ?) -- PARAMETERS: ["ORD_1dd3570d-b82e-4a8d-bb97-8a3b42766302","2023-09-17T21:50:06.434Z"]
INSERT INTO `item`(`id`, `quantity`, `orderId`, `productId`) VALUES (DEFAULT, ?, ?, ?), (DEFAULT, ?, ?, ?) -- PARAMETERS: [3,1,1,5,1,2]
COMMITPerfect! It actually works. Before we are done for today I want to talk about one more concept worth mentioning.
Understanding the Injection Chain
One thing to note here is request scoped providers propagate up in the injection chain. If we look at the injection chain for the orders module for example:
OrdersController → OrdersService → OrdersRepository
We can see that OrdersController depends on OrdersService, and OrdersService depends on OrdersRepository. However, just because OrdersRepository is request scoped, it will make anything else that depends on it request scoped as well. In that case, both OrdersController and OrdersService will also be request scoped. So technically, this approach will make almost all providers request scoped in our application. Even though this has a slight performance overhead, in most cases it’s negligible. However I thought it was worth mentioning that.
Conclusion
In this article, we have examined a simple and an advanced way of dealing with transactions. The advantage of the advanced way is that it prevents code duplication. Whenever the transaction interceptor is used on a route, if repository classes are implemented properly, the queries will automatically run in a transaction. I hope this article was useful to fellow developers out there. Happy coding 😃