API payloads validation and transformation in NestJS
This article focuses on techniques for the implementation of secure and error-proof APIs as much as it could be achieved by controlling API’s input and output payloads. This includes the following procedures:
- input deserialization, filtering, and validation
- output filtering, transformation, and serialization
Input procedures ensure that data getting into our system is in the correct format, does not contain harmful extra data, and is valid.
Output procedures ensure that the data we expose as output does not contain unwanted values (passwords for example) and is in the right format and structure.
Why we need DTOs
Although neglected in many tutorials DTOs are very useful in the real-world and actually, the rest of the tutorials make use of them. While adding some extra coding effort DTOs significantly increase the readability of your code and the stability of your API contract. Implementing predictable serialization/deserialization and validation with DTOs is easier as well.
DTO design tips
As you build out features like CRUD (Create/Read/Update/Delete) it’s often useful to construct variants on a base entity type. NestJS provides several utility functions that perform type transformations to make this task more convenient.
When building input validation types (also called DTOs), it’s often useful to build create, and update variations on the same type. For example, the create variant may require all fields, while the update variant may make all fields optional.
PartialType
NestJS provides the PartialType() utility function to make this task easier and minimize boilerplate.
The PartialType() function returns a type (class) with all the properties of the input type set to optional. For example, suppose we have a create type as follows:
export class CreateUserModel {
email: string;
password: string;
posts: Post[];
address: string;
}By default, all of the fields in this the above model are required. To create a type with the same fields, but with each one optional, use PartialType() passing the class reference (CreateUserModel) as an argument:
export class UpdateUserModel extends PartialType(CreateUserModel) {}OmitType
The OmitType() function constructs a new type or class by picking all properties from an input type and removing certain attributes. Consider the below example.
export class UpdateUserModel extends OmitType(CreateUserModel, ['password'] as const) {}Basically, the new model will have all the properties from CreateUserModel except the password.
PickType
The PickType() function constructs a new class or type by picking a particular set of properties from the input type:
export class UpdateUserPasswordModel extends PickType(CreateUserModel, ['password'] as const) {}Here we will have a new type that only contains the password attribute.
IntersectionType
The IntersectionType() function combines two types into one class or type.
Let’s look at an example where we have two models:
export class CreateUserModel {
email: string;
password: number;
posts: Post[];
address: string;
}export class AdditionalUserModel {
age: number;
}
Now we can create a new type as below.
export class UpdateUserModel extends IntersectionType(CreateUserModel, AdditionalUserModel) {}Input data deserialization and validation
Data coming over the wire is in string format by default. However, we often need to change it to another type before our application logic can use it. Hence, deserialization becomes an important aspect of any application. Also, we must validate user input, to ensure the input meets our app requirements and the user gets the best user experience and app functionality.
Validation annotations
With NestJS ValidationPipe, handling several validations becomes much easier. Let’s look at how to use the in-built ValidationPipe.
As a first step, we bootstrap the ValidationPipe in the bootstrap function available in the main.ts file.
See the below example:
import { ValidationPipe } from '@nestjs/common';
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';async function bootstrap() {
const app = await NestFactory.create(AppModule);
app.useGlobalPipes(new ValidationPipe())
await app.listen(3000);
}
bootstrap();
Here, we call the useGlobalPipes() method. Basically, we pass it as an instance of the ValidationPipe class. Notice here that ValidationPipe is part of the @nestjs/common package.
To test our pipe, let’s move to validation annotations and create a simple endpoint.
@Post()
create(@Body() body: CreateUserModel) {
return 'This action creates a user';
}Now we can add a few validation rules to our CreateUserModel. We do this using decorators provided by the class-validator package, described in detail here. In this fashion, any route that uses the CreateUserModel will automatically enforce these validation rules.
import { IsEmail, IsNotEmpty } from 'class-validator';export class CreateUserModel {
@IsEmail()
email: string;@IsNotEmpty()
password: string;
}
With these rules in place, if a request hits our endpoint with an invalid email property in the request body, the application will automatically respond with a 400 Bad Request code, along with the following response body:
{
"statusCode": 400,
"error": "Bad Request",
"message": ["email must be an email"]
}Let’s assume our CreateUserModel has a new field — post, which is an object and has multiple properties, to perform their validation too, then we need to use the @ValidateNested() decorator.
import { IsEmail, IsNotEmpty } from 'class-validator';export class CreateUserModel {
@IsEmail()
email: string;@IsNotEmpty()
password: string;
@ValidateNested()
post: Post;
}
The above code won’t work because when we are trying to transform objects that have nested objects, it’s required to know what type of object we are trying to transform. Since Typescript does not have good reflection abilities yet, we should implicitly specify what type of object each property contains. This is done using the @Type() decorator from the class-transformer package.
import { IsEmail, IsNotEmpty, ValidateNested } from 'class-validator';
import { Type } from 'class-transformer';export class CreateUserModel {
@IsEmail()
email: string;@IsNotEmpty()
password: string;
@ValidateNested()
@Type(() => Post)
post: Post;
}
If we want to validate an array of objects, pass { each: true } to the @ValidateNested decorator.
import { IsEmail, IsNotEmpty, ValidateNested } from 'class-validator';
import { Type } from 'class-transformer';export class CreateUserModel {
@IsEmail()
email: string;@IsNotEmpty()
password: string;
@ValidateNested({ each: true })
@Type(() => Post)
posts: Post[];
}
Filtering
We can also strip unwanted properties from incoming requests. In other words, we can whitelist certain properties. See the below address field.
import { IsEmail, IsNotEmpty, ValidateNested } from 'class-validator';
import { Type } from 'class-transformer';export class CreateUserModel {
@IsEmail()
email: string;@IsNotEmpty()
password: string;
@ValidateNested({ each: true })
@Type(() => Post)
posts: Post[];
address: string;
}
To achieve so, we can simply add another configuration parameter in ValidationPipe in the main.ts file as below:
app.useGlobalPipes(
new ValidationPipe({
whitelist: true,
}),
);Auto-transformation
Payloads coming in over the network are plain JavaScript objects. The ValidationPipe can automatically transform payloads to objects typed according to their model classes. To enable auto-transformation, set transform to true. To enable this behavior globally, set the option on a global pipe:
app.useGlobalPipes(
new ValidationPipe({
transform: true,
}),
);With the auto-transformation option enabled, the ValidationPipe will also perform conversion of primitive types. In the following example, the findOne() method takes one argument which represents an extracted id path parameter:
@Get(':id')
findOne(@Param('id') id: number) {
console.log(typeof id === 'number'); //=> true
return 'This action returns a user';
}By default, every path parameter and query parameter comes over the network as a string. In the above example, we specified the id type as a number (in the method signature). Therefore, the ValidationPipe will try to automatically convert a string identifier to a number.
Transformation pipes
We showed how the ValidationPipe can implicitly transform query and path parameters based on the expected type. However, this feature requires having auto-transformation enabled.
Alternatively (with auto-transformation disabled), you can explicitly cast values using the ParseIntPipe or ParseBoolPipe (note that ParseStringPipe is not needed because, as mentioned earlier, every path parameter and query parameter comes over the network as a string by default).
@Get(':id')
findOne(
@Param('id', ParseIntPipe) id: number,
@Query('sort', ParseBoolPipe) sort: boolean,
) {
console.log(typeof id === 'number'); // true
console.log(typeof sort === 'boolean'); // true
return 'This action returns a user';
}Output data serialization
It is a process of preparing an object to be sent over the network to the end client. To prepare an object could be to exclude some of its sensitive or unnecessary properties or add some additional ones.
For example, sensitive data like passwords should always be excluded from the response. Or, certain properties might require additional transformation, such as sending only a subset of properties of an entity. Performing these transformations manually can be tedious and error-prone, and can leave you uncertain that all cases have been covered.
Exclude
Let’s assume that we want to exclude a password property from a user entity automatically. We annotate the entity as follows:
import { Exclude } from 'class-transformer';export class UserEntity {
id: number;
firstName: string;
lastName: string;@Exclude()
password: string;constructor(partial: Partial<UserEntity>) {
Object.assign(this, partial);
}
}
Now consider a controller with a method handler that returns an instance of this class.
@UseInterceptors(ClassSerializerInterceptor)
@Get()
findOne(): UserEntity {
return new UserEntity({
id: 1,
firstName: 'Kamil',
lastName: 'Mysliwiec',
password: 'password',
});
}When this endpoint is requested, the client receives the following response:
{
"id": 1,
"firstName": "Kamil",
"lastName": "Mysliwiec"
}Note that the interceptor can be applied application-wide. In case we want to have a global serializer — we may use the useGlobalInterceptors method:
app.useGlobalInterceptors(new ClassSerializerInterceptor(app.get(Reflector)));The combination of the interceptor and the entity class declaration ensures that any method that returns a UserEntity will be sure to remove the password property. This gives you a measure of centralized enforcement of this business rule.
Expose
You can use the @Expose() decorator to provide alias names for properties or to execute a function to calculate a property value (analogous to getter functions), as shown below.
@Expose()
get fullName(): string {
return `${this.firstName} ${this.lastName}`;
}Transform
You can perform additional data transformation using the @Transform() decorator. For example, the following construct returns the name property of the RoleEntity instead of returning the whole object.
@Transform(({ value }) => value.name)
role: RoleEntity;Serialize options
You may want to modify the default behavior of the transformation functions. To override default settings, pass them in an options object with the @SerializeOptions() decorator.
@SerializeOptions({
excludePrefixes: ['_'],
})
@Get()
findOne(): UserEntity {
return new UserEntity();
}Options passed via @SerializeOptions() are passed as the second argument of the underlying classToPlain() function. In this example, we are automatically excluding all properties that begin with the _ prefix.
References
You can find more information in the official documentation of NestJS, and underlying packages class-validator and class-transformer.
