Crafting a Persistent Gmail Sync with NestJS and OAuth2

Photo generated by Bing

This is fifth story of series Building a Robust Backend: A Comprehensive Guide Using NestJS, TypeORM, and Microservices. Our purpose is to build an email sync system for Gmail, with environments, migrations and everything set, we can proceed with setting up Gmail sync. setup.

We will need to register a test app with Google account, setup oAuth2 on it and define scope in it and lastly get the service file as JSON. Since there are already many tutorials available so I won’t be repeating one, but for the App scope, set the scope according to the image below. As for redirect URI, set it http://localhost:5000/api/v1/gmail-accounts/webhook. And as for emails, we can’t just get authentication for any app in test mode, we need to define some emails that we will be using for testing, so define at least two emails for better testing.

Gmail app scope

Now we have the JSON file that will look something like below.

{
  "web": {
    "client_id": "some_id",
    "project_id": "some_id",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_secret": "some_secret",
    "redirect_uris": ["http://localhost:5000/api/v1/gmail-accounts/webhook"],
    "javascript_origins": ["http://localhost:5000"]
  }
}

Let’s create APIs for registering users and setting up oAuth2. Install the google api package from npm using npm i googleapis. Create constants file in shared/utils and add credentials path and Gmail creds path in it. Though we will be placing Gmail creds file in utils/credentials folder but it's better to assign path as variable, so we won’t need to make many changes in future. Add JSON files in credentials in .gitignore so our sensitive data won’t go to repo or public by adding following lines in file:

# Credentials
/src/shared/utils/Credentials/*.json

We have the credentials JSON file but its sole purpose is to provide OAuth2 object, the Oauth2 object can be new or derived from existing token, Since it’s a common file that can be required in different modules, so let’s make a utility function in shared folder named getOAuthClient and utilizie it where ever we need the google OAuth for different operations. Code for functiongetOAuthClient is:

import * as fs from 'fs';
import { CREDENTIALS_PATH, GMAIL_CREDS } from '../utils/constants';
import { google } from 'googleapis';
import { OAuth2Client } from 'google-auth-library/build/src';

export default function getOAuthClient(
  token: object | null = null,
): OAuth2Client {
  const content = fs.readFileSync(CREDENTIALS_PATH + GMAIL_CREDS, 'utf-8');
  const credentials = JSON.parse(content);
  const { client_secret, client_id, redirect_uris } = credentials.web;

  const oAuth2Client = new google.auth.OAuth2(
    client_id,
    client_secret,
    redirect_uris[0],
  );
  if (token) {
    oAuth2Client.setCredentials(token);
  }

  return oAuth2Client;
}

Update the create method in services to send oAuth2 link to user after storing user data in DB, the updated create method code is:

  async create(dto: CreateGmailAccountDTO): Promise<string> {
    const gmailAccount = this.gmailAccountRepository.create(dto);
    await this.gmailAccountRepository.save(gmailAccount);
    // Added OAuth2 call for getting user email access
    const oAuth2Client = getOAuthClient();
    const authUrl = oAuth2Client.generateAuthUrl({
      access_type: 'offline',
      scope: ['https://mail.google.com/'],
      state: dto.email,
    });
    return authUrl;
  }

One thing that we have done is not coherent to our existing APIs structure and that is setting redirect URI to http://localhost:5000/api/v1/gmail-accounts/webhook event though we don’t have api/v1 as any route. By convention we define api as route part to tell that it is an API route and v1 is used for future convention in production to maintain backward compatibilities i.e., in case of API upgrade, we can assign it to v2 rather than defining some absurd route. At the moment since our whole application is on v1 so we will set it as global prefix by adding app.setGlobalPrefix(‘api/v1’); in main.ts.

With routing in clear let's create a webhook listener to listen to the Google auth response and storing the API keys. Just to clarify, on webhook we receive code that we further use to get token and the code receiving part in future will be handled by SPA application like react etc. So, we will get the code, and call get token method to get token against received code. The token will actually be an object containing access_token, refresh_token, scope, expiry_date and token_type. Below code extracts the data and updates the DB.

  async getWebhook(query: { code: string; state: string }) {
    const oAuth2Client = getOAuthClient();
    const { tokens } = await oAuth2Client.getToken(query.code);

    const gmailAccount = await this.gmailAccountRepository.findOne({
      where: { email: query.state },
    });

    const updates = {
      access_token: tokens.access_token,
      refresh_token: tokens.refresh_token,
      scope: tokens.scope,
      expiry_date: tokens.expiry_date,
      token_type: tokens.token_type,
    };

    await this.gmailAccountRepository.save({
      ...gmailAccount, // spread the existing account
      ...updates, // spread the new updates
    });

    // Return some response or the updated gmailAccount
    return { message: 'Gmail account updated successfully.' };
  }

Now with the service method in place, we will an endpoint to listen to GET request on webhook URL. Let’s delete the existing DB record and create API call, it will respond with URL as shown in image below. Open this URL in browser and allow application access from test email. Word of caution, it won’t work with emails that are not added as test so you will get error on these emails auth screen.

API response with URL

Now one issue left is that this token is valid for around an hour only, so after an hour do we need to login again? No, we will use refresh token to get new token without requiring signing into our email again. Though we won’t be using it now, we will need it in the future, so let’s set it up. We will need to get token from DB, check if token is expired from its expiry date, refresh the token if its expired and update the token in DB if it is expired. Below are the functions achieving this.

  // Function to check if the token is expired
  public isTokenExpired(tokenExpiryDate: number): boolean {
    return tokenExpiryDate <= Date.now();
  }

  // Function to refresh the token
  public async refreshToken(token: any): Promise<any> {
    const oAuth2Client = getOAuthClient(token);
    const refreshedTokenResponse = await oAuth2Client.refreshAccessToken();
    return refreshedTokenResponse.credentials;
  }

  // Function to update the token in the database
  public async updateTokenInDB(id: string, updatedToken: any): Promise<void> {
    const user = await this.gmailAccountRepository.findOne({ where: { id } });
    if (user) {
      await this.gmailAccountRepository.update(user.email, updatedToken);
    }
  }

  // Function to validate the token and get an updated one if needed
  public async validToken(id: string): Promise<any> {
    const token = await this.getToken(id);
    if (!token) {
      return null;
    }

    if (this.isTokenExpired(token.expiry_date)) {
      const credentials = await this.refreshToken(token);

      const updatedToken = {
        access_token: credentials.access_token,
        refresh_token: credentials.refresh_token || token.refresh_token,
        expiry_date: credentials.expiry_date || token.expiry_date,
        token_type: credentials.token_type || token.token_type,
      };

      await this.updateTokenInDB(id, updatedToken);
      return updatedToken;
    }

    return token;
  }

  // Get token from DB
  public async getToken(id: string): Promise<{
    access_token: string;
    refresh_token: string;
    scope: string;
    expiry_date: number;
    token_type: string;
  } | null> {
    const user = await this.gmailAccountRepository.findOne({
      where: { id: id },
    });

    if (user) {
      return {
        access_token: user.access_token,
        refresh_token: user.refresh_token,
        scope: user.scope,
        expiry_date: user.expiry_date,
        token_type: user.token_type,
      };
    }

    return null;
  }

Now with the token refreshing handled, our test app account setup is complete. However, we have done several mistakes till now that we need to modify before proceeding, our responses are inconsistent, and our functions aren’t properly typed so in next story we will address these issues before proceeding. As usual, this story code is available on GitHub in feature/integrate-gmail-oAuth branch. If you appreciate this work, please show your support by clapping for the story and starring the repository.

Before we conclude, here’s a handy toolset you might want to check out: The Dev’s Tools. It’s not directly related to our tutorial, but we believe it’s worth your attention. The Dev’s Tools offers an expansive suite of utilities tailored for developers, content creators, and digital enthusiasts:

• Image Tools: Compress single or multiple images efficiently, and craft custom QR codes effortlessly.
• JSON Tools: Validate, compare, and ensure the integrity of your JSON data.
• Text Tools: From comparing texts, shuffling letters, and cleaning up your content, to generating random numbers and passwords, this platform has got you covered.
• URL Tools: Ensure safe web browsing with the URL encoder and decoder.
• Time Tools: Calculate date ranges and convert between Unix timestamps and human-readable dates seamlessly.

It’s a treasure trove of digital utilities, so do give it a visit!