How I built an email classifier service using GPT-4o and Langchain

In another episode of using LLMs for basic ML tasks. I recently got a full-stack assignment which I had to complete in 3 days.

The task was simple: Create a website to allow users to log in via their Google Account, fetch their ‘X’ recent emails from Gmail, and classify them using GPT-4o with langchain.js

Now I have been told that I yap a lot in these blog posts so I will keep it sweet, simple and to the point.

Task 1: Authentication

Previously I used Auth0 and had an excellent experience with it too but I wanted to try something new(a reference to my trend of kicking myself in the balls) so I chose Supabase which I have heard much about but never used before.

I don’t want any smoke or it may be my skill issue speaking but setting up Google OAuth and working with Supabase in my Next JS code was quite confusing tbh. The docs and YouTube videos showed either supabase.jsor supabase/ssrand some blogs and StackOverflow answers used supabase/auth-helperswhich confused me about what I should use.
After much reading and research, I came to know that supabase/auth-helpers is a legacy library which is depreciated in favour of supabase/ssr which is essentially a wrapper on top of supabase.js(which I came to know after looking into the method definitions of ssr package).

In short, I have to use the ssr library.

For Auth, I had to make 2 util files, client.ts which is used for any functionality I want to handle on the client side and server.ts which is used for any functionality I want to handle on the server side(get, set and remove supabase auth cookies).

For detailed implementation, click on the image

Now was the “simple” part, I had to make a Login with Google button which will use that client.ts function to signInWithOAuth and I had to give the access_type , prompt and scope (yes you have to give scope here which will appear on your consent screen during login). Ofc the prerequisite for doing all this would be to enable Google OAuth in Supabase and Google Console Cloud and follow the relevant instructions.

In the backend, I had to make a GET request route which will handle the callback after the user completes their sign-in and is sent back to my website. The URL after login looks something like this https://checkemails.vercel.app/api/auth/callback?next="%2Femails"&?code="xxxx"

For OAuth login, I used the PKCE flow which is a more secure version and by using exchangeCodeForSession I get the provider and refresh token which I can use in the Gmail API in the next step.

To access any User Information from Supabase

Task 2: Fetching the emails using Gmail API

After working with modern API and technologies, I realised that I had been spoiled by their easy-to-onboard docs cause there were so few docs for the Gmail API for node.js that it is not even funny. After much searching, I did find a very obscure but good video about using Gmail API but it was using direct fetch requests and not googleapis to get data. I had to figure out how to convert that fetch request to googleapi request by myself.

My experience with Gmail API docs

Realistically there is not much advantage of googleapis over fetch except that it gives a proper interface and types which is very essential for my app cause i’m typing a script(pun intended).

The main problem is that for basic information like “From” or “snippet” you can easily get them by just accessing the keys from the request but if you want to re-create the body of the email like it’s shown in your Gmail website then that's a whole another story. For that, I made a neat little function to re-create the body from message.payload and return plainText and htmlText .

Re-constructing the body of the email

Now I got an object to return which contained {from, snippet, htmlText, plainText} and I used that to render beautiful HTML emails with React Letter

Task 3: Classification using Langchain

I have never used AI in any project because it usually costs money. So my approach has been to first code the entire part without the OpenAI API Key. Then, I go to the playground to work on prompt engineering. Once I have a decent prompt, I plug it into my code along with the API key and viola!

Had to ask my father to credit my OpenAI account since Stripe doesn’t take RuPay cards :(

Although I did question this part of the assignment a little bit since an LLM might be overkill just to classify some emails but I understand it is for the sake of assignment and its not a ML internship

The first step would be getting the API key from the user. I made a modal which verifies the key if it is a valid OpenAI key and saves it in the LocalStorage if it is.

To verify the validity, I used a neat little trick. I used the OpenAI SDK to list the available models and if the key is not valid then that would throw an Authentication error.

Using my meagre ML/Data Science knowledge, I knew that before training any data, we should preprocess it. For each email, I have 2 types of content viz. plainText and htmlText .
For context, plainTextcontains the normal text inside the email and htmlTextis the HTML code which is used to make those beautiful HTML Emails. To process the plainText I had to remove all kinds of links CSS styles, HTML tags, and non-ASCII characters and normalise whitespace characters using a long function.
Then I would have to process htmlText for which I used the html-to-text library for the initial run and then replaced all whitespace characters with a single space, removing non-printable and non-ASCII characters and trimming the text.

I had to minimize the email data without losing its semantic meaning so that fewer tokens would be used. I used OpenAI tokenizer to get an estimate of how many tokens is the prompt email content taking and had to find a sweet spot. The most frustrating part while cleaning the data was dealing with non-printable, non-ASCII characters cause well…they are invisible and each one takes a single token thus maximising cost.

The reason why I had to do almost the same pre-processing on both htmlText and plainText is because I cannot trust the sender of the email or Gmail and it was also because I did all kinds of exploratory analysis on my data until I got it in the form which I wanted. In the end, I had an array of JSON objects containing the index and contents of the emails.

Then the next part was the actual AI part. I did an oopsie here and didn’t read the assignment description properly and did the AI part using openai package directly BUT on the last day of submission, I realised in the morning that I did in fact have to use Langchain for the classification.

My strategy for prompting was few-shot learning with zero temperature.

Few-shot learning is basically where I give examples in my prompt of email and their category after classification so that the AI can give more accurate answer when I ask with the user’s email data.
Temperature basically defines the degree of randomness in the answer. Temperature closer to 0 will give more deterministic and repetitive answers.

Using Langchain, I first created an Example Prompt Template and an example array of JSON objects which contained index and email contents.
One thing I had to ensure was that the examples and the JSON email data I would provide via prompt were in the same format.

Using the example prompt template, examples array and my actual prompt, I created a few-shot learning prompt using Langchain methods.

I then created a message array which contained the System prompt and User prompt. One thing I had to make sure of was that the returned output from the AI should be in the JSON format so that I could easily parse it and send it back to the frontend. Thankfully, Langchain also provides a method for that, JsonOutputParser . I then invoked the model using my prompt and piped it to the parser to get an output in JSON format.

The code is a little long and my blog is almost complete so please bear with me.

But AI is unpredictable at times and it may happen that the model will not give me the answer properly in the expected JSON format. For that reason, I did another neat little retry logic in case an error happens the first time.

I sent the data back to the frontend properly formatted it and showed it on my frontend. Thus the assignment has been completed. There are many other things which I have avoided explaining here but I may explain it in detail if I ever make a video. You can also check out the assignment here:

Check.Emails

Congratulations🎊, you have reached the end of my blog post. Thank you for staying patient and reading through my journey of a 3-day assignment. I know that the things I did in my assignment can surely be done in a better way but I’m still learning and will eventually improve.

My honest thoughts after completing the assignment were that I’m really thankful to the person for creating such a cool assignment cause I learned so many things while doing it.
Follow me on Twitter for regular updates and thank you for reading.