Using Zod To Validate Data

Validating External Data in a Full-Stack Next.js Application

When building a full-stack application, it’s common to encounter scenarios where external data needs to be integrated into your app. Whether fetching data from a server or dealing with user inputs, the data received may not always conform to expected shapes, potentially leading to bugs. While TypeScript provides static typing, it alone may not be sufficient. In this article, we’ll explore ten instances where careful validation of external data is crucial and discuss why using a schema validator like Zod is beneficial.

Understanding Full-Stack Development

A full-stack application typically comprises a front end and a back end. In this discussion, we’ll focus on React and Next.js for the front end, but the principles apply to other technologies.

Let’s consider a full-stack Next.js application. The front end may fetch data from the back end, either from its own or third-party APIs. External data can also come from user inputs through forms, local storage, or even the URL.

Front End: Sources of External Data

1. API Requests to Back End: The front end often makes API requests to its own back end to retrieve data. This data may be crucial for various features, such as product information in an e-commerce application.
2. Third-Party APIs: External data can also be sourced from third-party APIs. Customizing the website based on user location using a third-party API is a common scenario.
3. User Inputs (Forms): User-filled forms are another source of external data. Validating form data is essential to ensure it aligns with the expected structure.
4. Local Storage: Storing and retrieving data from local storage requires validation to handle potential variations in the stored data.
5. URL Parameters: Data stored in the URL, such as search parameters, should be validated before being processed within the application.

Why TypeScript Alone Falls Short

While TypeScript provides static typing and enhances code clarity, it doesn’t guarantee that the data received matches the expected schema. External sources introduce uncertainty, making validation crucial.

Introducing Zod for Schema Validation

To address these challenges, a schema validator like Zod can be employed. Zod allows developers to define schemas for their data and validate that incoming data conforms to those schemas.

In the next section, we’ll delve into how schema validation works on the front end using Zod. Stay tuned for practical examples and insights into enhancing data integrity in your full-stack Next.js application.

Validating External Data in Full-Stack Next.js Application

In this segment, we’ll explore a critical scenario in full-stack Next.js application development — validating external data. The focus here is on API requests, and we’ll discuss why TypeScript alone falls short in ensuring data integrity, necessitating the use of a schema validator like Zod.

Example 1: API Requests

Consider a common scenario where the front end makes API requests to its own back end or a third-party API. Let’s look at a simple product component fetching data from the back end.

In Next.js, there are client and server components, and while server components can fetch data, there are cases where fetching from the client is necessary. The example employs the useEffect hook for data fetching, making a request to a backend route that returns a product object.

The issue arises when TypeScript, by default, types the API response as any, providing no type safety. This lack of type safety becomes problematic when the shape of the data changes on the back end, leading to potential crashes in the application.

Misalnya kita membuat backendnya seperti ini:

Kemudian di bagian frontend kita melakukan useEffect untuk mendapatkan datanya:

Jika kita hover typescript mereturn value dari API bertipe data ANY

Sekarang bagaimana jika di real casenya user tidak mengirimkan nama tapi justru mengirimkan id:

Sekarang aplikasi kita mengalami error:

Jika kita menambahkan tipe data nya pun ini tidak menjadi solusi

Typescript disini akan tetap mereturn error

Walaupun kita sudah menambahkan tipe datanya. Karena typescript itu tidak ada ketika run time.

Kita inginnya robust application.

Kita bisa mensolvenya dengan menambahkan optional chaining ?

Akan tetapi bagaimana jika data yang dikirimkan kosong?

Tentu kita perlu menambahkan optional chaining lagi:

Jika kamu lihat kita tidak mendapatkan error pada layar:

Sekarang bagaimana jika price disini tidak dalam bentuk format angka:

Dan kita menambahkan opsional chaining lagi ?

Ternyata hasilnya error. Karena toFixed tidak ada untuk data yang string.

Jadi solusinya adalah kita perlu cek tipe datanya:

Challenges with TypeScript Alone

Even when attempting to type the product variable with TypeScript, the dynamic nature of runtime data fetching poses challenges. If the back end changes the data structure, TypeScript won’t catch these issues during runtime, leading to runtime errors in the application.

Optional chaining might mitigate some issues, but it falls short in providing robust protection, especially when dealing with more complex data structures or when specific methods are expected.

Introducing Zod for Schema Validation

To address these challenges, a schema validator like Zod comes into play. Zod allows developers to define schemas for expected data structures and validates incoming data against these schemas. This ensures that the data adheres to the expected structure, preventing runtime errors and enhancing overall robustness.

Zod merupakan run time depedency bukan development depedency seperti typescript.

Yang perlu kita lakukan adalah membuat skema — ekspetasi kita terhadap data yang kita terima.

Dengan parse, ketika kita mendapatkan input yang tidak sesuai dengan skema maka akan menthrow error. Atau kita bisa menggunakan safeParse, dimana dia tidak akan menthrow error, hanya mereturn success properti:

Misalnya kita mengirim id alih-alih mengirim nama:

Kita mendapatkan error message dari Zod.

Jika datanya benar:

Pada dasarnya jika kita ngambil dari API kita engga tau tipe datanya apa, sehingga kemungkinan kita akan memberi tipe data UNKNOWN ke typescript:

Nah gimana kalo kita mau buat fungsi yang mengambil harga dari produt seperti di bawah:

Kita perlu menambahkan tipe datanya lagi bukan? Padahal kita sudah punya schema yang sama di Zod dengan typescript.

Kita bisa melakukan ini agar tidak mendefinisikan ulang:

Canggihnya adalah, jika kita mendefinisikan tipe datanya, maka akan automatis di typescriptnya tersinkronisasi:

Implementing Zod

To use Zod, developers define a schema, specifying the expected structure of the data. In the example, a product schema is created using z.object with expectations for name (string) and price (number). The actual data fetched from the API is then validated against this schema using Zod's parse or safeParse methods.

Zod provides detailed error messages, making it easier to identify which part of the data structure doesn’t match the expected schema. By incorporating Zod, developers can confidently handle incoming data, reducing the risk of crashes and ensuring a more resilient application.

In practical scenarios, tools like React Query or TRPC may complement Zod for handling data fetching, caching, and more advanced scenarios, depending on the specific needs of the application.

In the next part, we’ll continue exploring other instances where validating external data is crucial in a full-stack Next.js application. Stay tuned for insights into handling user inputs, local storage, and URL parameters.

Handling User Input and Local Storage with Zod

Continuing our exploration of data validation in a full-stack Next.js application, we now turn our attention to scenarios involving user input and local storage.

Example 2: Form Data Validation

User input, particularly from forms, is a common use case for data validation. Imagine a checkout form where users provide information such as name, email, address, city, state, and ZIP code. Each field has its own specific data type (e.g., email is a string, ZIP code is a number), and checkboxes may represent boolean values.

Zod proves useful in this context as it allows for rigorous validation of user input. By defining a schema that describes the expected structure of the form data, developers can ensure that the data conforms to these expectations. This not only provides immediate feedback to users for correcting input errors but also sets the stage for consistent and secure data handling.

While tools like React Hook Form or Formik are commonly used for form management on the client side, Zod complements them by allowing developers to validate the same schema on both the client and the backend.

This becomes crucial when submitting form data to the backend, ensuring that the incoming data adheres to the same validation rules.

Integrating Zod with React Hook Form

To illustrate this integration, a Zod resolver is introduced.

By connecting Zod to React Hook Form through a resolver, developers can leverage Zod’s powerful schema definition capabilities for form validation. The example includes a sign-up schema with detailed specifications for email and password fields.

The benefits of using Zod in conjunction with React Hook Form include the ability to specify custom error messages, perform advanced validations (e.g., checking if passwords match), and infer TypeScript types directly from the schema. This tight integration allows developers to catch errors early in the development process, benefiting both frontend and backend validation efforts.

Incorporating Zod for data validation in various scenarios, including user input and local storage, contributes to the overall resilience and consistency of a Next.js application. The ability to use the same schema on both the frontend and backend ensures a unified approach to data validation, reducing the likelihood of errors and simplifying maintenance.

Example 4: localStorage Data Validation with Zod

In this example, the focus is on a shopping cart component, and the goal is to store cart information in local storage. Storing this information allows for a seamless user experience, as the cart data can be quickly retrieved when the user returns to the application. However, handling data from local storage poses challenges, particularly in ensuring the data’s structure aligns with expectations.

Misalnya kita mengammbil data dari localStorage, akan tetapi jika kita hover cart kita mendapatkan tipe data any.

Karena any sekarang kita bisa akses apa saja di dalamnya:

Nah untuk mengatasi masalah ini kita bisa menuliskan tipe datanya:

Atau jika kita tidak tau tipe datanya, kita bisa tuliskan unknown .

Tapi kemudian kita akan di peringatkan oleh typescript:

Sehingga mengharuskan kita membuat if di dalamnya:

Jadi zod bisa mengatasi masalah ini:

Sekarang kita bisa mengakses datanya:

Kita remove karena kita asumsikan kalo bentuknya udah berbeda dengan skema yang kita buat berarti localstoragenya sudah berbeda.

Initial Approach

The initial implementation involves retrieving the cart information from local storage using localStorage.getItem(). Since this data is expected to be in JSON format, it is parsed using JSON.parse(). However, this process could return null if the local storage is cleared. In such cases, the code attempts to set cart to an empty array or object, allowing for further operations.

Type Challenges with any and unknown

The TypeScript type system presents challenges in this scenario. Initially, the cart is implicitly typed as any, indicating that TypeScript assumes any property can be accessed. While this provides flexibility, it sacrifices type safety.

To address this, the code attempts to provide a more specific type, such as cart: CartType, where CartType might represent the expected structure. However, TypeScript's unknown type proves more suitable for situations where the exact structure is unknown.

Improving Type Safety with Zod

To enhance the type safety of the application and ensure the retrieved cart data adheres to a specific schema, Zod is introduced. A cardSchema is created using Zod, defining the expected structure of the cart data. This schema includes specifications for an array of objects, each having properties like ID and quantity. Notably, Zod allows for detailed specifications, such as requiring the quantity to be a positive integer.

Validating Data with Zod

The localStorage retrieved data is then passed through the cardSchema using Zod's validation capabilities. If the data adheres to the schema (success: true), it is considered validated and can be safely used. If validation fails, indicating a mismatch with the expected schema, the data is removed from localStorage to prevent using outdated or incompatible information.

Example 5: URL Data Validation with Zod

In this example, the focus is on handling data from the URL in a Next.js application. Often, data is stored in the URL, and it becomes necessary to read and validate this data before using it in the application. To achieve this, the useSearchParams hook in Next.js is introduced, providing access to search parameters in the URL.

Schema Definition with Zod

Before working with the data obtained from the URL, a schema is created using Zod. The searchParamsSchema is defined as an object, and within this object, expectations for certain parameters are set. For instance, an ID is expected in the URL, and using z.number, the schema ensures that this ID is of numeric type. Additionally, a color parameter is anticipated, and its validation is refined using Zod's z.enum to restrict it to values of "red," "green," or "blue."

Semua URL itu dibaca string, nah di zod kita bisa mengubahnya menjadi integer:

Kita dapat langsung mentranfrom atau mengubah idnya menjadi nomor:

Dan kita bisa menspesifikkan data warna yang kita dapatkan:

Dan kita bisa memvalidasinya:

Coercion and Transformation

One notable feature of Zod is its ability to coerce data. Since data retrieved from the URL is in string format, Zod can automatically transform it into the desired type. For instance, if the ID is expected to be a number, Zod can coerce it from a string to a number. This ensures consistency and eliminates the need for manual type conversions.

Validating URL Parameters with Zod

The data obtained from the URL, provided by useSearchParams, is then processed through the defined Zod schema. If the validation is successful (success: true), the data is considered safe to use. However, if validation fails, appropriate error handling can be implemented.

Improved Type Safety and Autocomplete

By incorporating Zod into the URL parameter validation process, type safety is significantly enhanced. TypeScript is now aware of the expected structure of the data, and developers benefit from autocompletion features when working with the validated data. This prevents common mistakes that beginners might make when dealing with pre-validation data.

Backend

Let’s now delve into the back end with a Next.js example. In Next.js, when creating a full-stack application, you can easily add backend functionalities.

This is done through the use of API route handlers. There are API route handlers, server actions, and server components, which are React components that only run on the server.

The question is, where will the backend get external data from? When we submit something from the front end to the back end, from the perspective of the back end, that’s incoming data.

Data coming from Frontend

Generally, data originating from the client cannot be fully trusted. Therefore, when we receive data from the front end, we want to process it through Zod or a more robust setup like trpc.

For example, form data will be obtained on the front end and run through Zod. If there are issues during parsing, we can immediately provide feedback to the user for correction.

If everything is good, we willingly accept it into our front-end application and likely submit it to our server. On the server side, when we receive that data, we’ll parse it again, and with Zod, these schemas can be reused both on the client and server sides.

Third-party API

Typically, when submitting form data in Next.js, you work with it in API route handlers or server actions. However, there are other sources of external data for the backend as well.

For instance, the backend may make third-party API requests itself. When the backend gets data from a third-party API, we want to run it through Zod first, as we don’t control the backend of that third-party API.

Webhooks

Another source of external data for the backend is webhooks. For example, in payment scenarios, if someone successfully completes a payment with Stripe, Stripe will send a message notifying you of the successful payment and include relevant data. Before working with that data, we would run it through Zod first. These are all network requests, whether from the front end, third-party APIs, or webhooks.

Environment Variables

There are other sources of data for the backend as well, such as loading environment variables into your app. When working with environment variables, you also want to ensure they are loaded properly into your app, which can be done with Zod.

File system, URL, database

Similarly, data from the file system, URL, or database on the backend can be run through Zod to ensure reliability and security.

Practical Case in The Backend

Misalnya kita mempunyai API handler seperti gambar di atas, kemudian kita akan mengambil data json-nya.

Tentu tipe datanya merupakan unknown karena kita gatau data yang datang apa.

Kemudian tepat dibawahnya kita bisa melakukan validasi data secara manual, tapi itu tidak disarankan.

Lebih baik menggunakan Zod. Tapi kita tidak mau membuat skema secara berulang-ulang. Kita inginnya reusable.

Kita bisa mencapai ini dengan folder lib, dan di dalam folder tersebut terdapat 2 files:

• types.ts
• validation.ts

Sekarang kita bisa import di front end dan backend

Kemudian:

Dan karena kita mempunyai one source untuk validasi, maka kita tidak perlu dua kali untuk mengaplikasikan validasi ini di backend atau frontend. Cukup ubah source datanya saja.

Dan dibagian skema kita tidak perlu membuat ulang skemanya:

This API is designed to receive data from the front end, such as form submissions. The tutorial emphasizes the need to handle and parse the incoming data effectively.

They are in a route handler where the form data will be received. They highlight the importance of inspecting the request body to parse it correctly. Since TypeScript defaults the body type to ‘any,’ so we are using ‘unknown’ to be more cautious, considering the uncertainty about the incoming data structure.

To avoid messy and unstructured validation with numerous if statements, we use of Zod for data validation. The suggestion is to reuse a schema that can be imported from a library folder containing all the schemas. A specific example is given, where a checkout form schema is created with various properties like name, email, phone number, and address, each with optional custom error messages.

The tutorial demonstrates how this schema can be imported both on the front end and back end. On the back end, the incoming form data is passed through the Zod schema for validation. If the data does not conform to the expected shape, the server responds with a JSON containing errors and a status code of 422 (indicating malformed data). If successful, the tutorial suggests that the data can be further processed or stored in a database.

The advantage highlighted is the centralized nature of the schema. Any changes made to it immediately reflect on both the front end and back end, ensuring consistency. The tutorial also briefly mentions using Zod for validating incoming data from third-party APIs or web hooks, emphasizing the importance of validation when the server does not control the data source.

Environment Variable Case

Alih-alih kita pake process.env, kita tidak akan mendapatkan saran autocompletion.

Kita menggunakan parsedEnv yang kita dapatkan dari konfigurasi schema env kita.

We need to create a TypeScript file to define a schema for expected environment variables. This approach involves using the Zod library to create a schema that validates the structure and types of environment variables. The schema is designed to match the object structure of process.env, a Node.js environment variable that contains key-value pairs. The example includes validations for variables such as databaseURL (expected to be a non-empty string), port (possibly a number), and a third-party API key (a string with a minimum length of one character). The parse method from Zod is recommended for this scenario, as it allows for immediate error detection and resolution during development. The resulting parsed environment variables can be exported for use throughout the application. The speaker demonstrates importing and using these variables in a route handler, emphasizing the benefits of autocomplete and the assurance that the variables have been loaded correctly. The article underscores the utility of Zod in managing and validating environment variables effectively.

File system case

Example 10 delves into handling JSON data in the local file system of a project. Imagine having a JSON file, named data.json, residing in the library. The speaker illustrates the scenario where access to this JSON data is required, such as within a route handler. Rather than hardcoding the product details, the goal is to dynamically retrieve the data from the data.json file.

The speaker walks through the process, starting with importing the path module and using fs.readFile to read the contents of the JSON file. Importantly, the promises version of the file system module is utilized for asynchronous file reading. However, since the structure of the file contents is uncertain, the speaker recommends running it through a product schema, ensuring it conforms to expectations.

This product schema, similar to those used in previous examples, validates the presence of attributes like ID, name, and price, and checks if the price is a number. This validation ensures that the data obtained from the file system aligns with the anticipated structure.

The example underscores the importance of schema validation not only for external inputs but also for data retrieved from the local file system. It provides a structured and reliable approach to handling JSON data, promoting consistency and reducing the risk of unexpected issues arising from mismatches in data shapes.

URL case

Database