OAuth 2.0 with Google Client Libraries: Java SDK
Disclaimer: This implementation guide aims to enhance one’s understanding of the concept and not to building a full fledged Gmail scraping application and hence it has very limited operations demonstrated to be performed on a user’s mailbox.
Enough with the setup and theories about Google OAuth in our last discussion here:
Let us now enjoy some action !
Surprise !! At least to me it was. To be honest, I thought I had committed a lot stating that I’ll be back with the complete implementation of the Google OAuth 2.0 Gmail use case, in my last article. Turns out, I was selling myself way too less to myself.
Yes, I am back with a complete guide to implementing OAuth 2.0 for your Gmail scraping application. So, just to reiterate, this guide takes Gmail as a use case of Google OAuth 2.0 implementation and uses Gmail API SDK for implementation.
Now, for the smart ones reading this, if you are already wondering that you’ll have to refer to this article every time you want to copy some lines of code from the implementation, do not worry, I have got you covered. Instead of referring to the code snippets in this article you can simply have the implementation cloned from my Git repo.
You’re welcome.
So let’s jump right into it, shall we?
Again, what ?
Ok, so this is for the ones who still, even after “going through” the earlier mentioned article and Github repository, did not get an idea of what it is actually we are talking about.
This is all about securing the connection to a third party email server for any cloud application that tries to automate workflows by scraping mail servers and extracting information on behalf of its clients.
This is done by OAuth 2.0 protocol. The “how to setup” part is what we discussed in the last article and the “how to implement” part is for this one. So to sum up, we will be seeing how to use Google’s OAuth 2.0 to connect to Gmail servers for a given user and operate on its inbox.
Making sense
As we must know by now, OAuth setup has some core components that need to be defined before leveraging the protocol. Let’s try to recall those:
- Client — In our case it is our Java application.
- Resource Server — In our case, it is Gmail.
- Resource Owner — In our case, it is the user whose inbox we intend to operate on.
- Authorization Server — In our case, it is Google.
Now, let’s weave these components together to make some sense:
As is clear from the above flow, the Client component needs to be smart enough to understand the responses from all other components and also to make a valid request to those components.
This smartness is nothing but the API SDKs provided by the third party service provider. In our case, we will use Google OAuth API SDK and Gmail API SDK. These APIs define a contract between the client and the server so that the client could make a valid request as well as understand a response that it gets from the server.
Google OAuth 2.0 Client
As we discussed already, we will be using the OAuth 2.0 client approach for implementing OAuth 2.0 for our use case. The other approach, which is using a Service Account, needs admin access to one’s organization’s Gmail domain.
For this client application, I have already created a corresponding OAuth 2.0 client in a public Google cloud console project. By “public” I mean that it is open for any user whose email address ends with “gmail.com” and is added to the project by the owner.
Client asks for permission to impersonate Resource Owner
Client application needs to get a grant from the resource owner which authorizes it to attempt requesting access and refresh tokens on behalf of the resource owner.
Here is how we can achieve this by the Google OAuth Java SDK:
Let’s break it down, shall we?
There are 2 constants that you need:
Then, we need to provide the client id and client secret of the OAuth 2.0 client that we created, to our client application. This is the identity of the OAuth client and is required for the Google authorization server to validate our client application. This line gets the JSON file downloaded from google cloud console while creating OAuth 2.0 client:
Here are some additional important information that our client application will get from reading this JSON file:
- auth_uri:”https://accounts.google.com/o/oauth2/auth"
- token_uri:”https://oauth2.googleapis.com/token"
We will see eventually where these URIs are used, but for now let’s initiate the authorization flow:
Here, we use the OAuth 2.0 client to initiate the authorization flow with some added properties. An important thing to notice here is the use of DataStoreFactory. This data store is for storing the access and refresh tokens for the user, so that even if the application gets terminated, the next time the application starts, it wont start from scratch.
Now, we need to open a port that will listen to the grant (nothing but the authorization code) from the resource owner. The following will open port 8888:
If you are thinking that something is missing, we did not use the “auth_uri” yet, here is how it is used:
Let’s look closely, what we might not have observed is the use of auth_uri. Once the above code line executes, a URL is printed in the System output or the console:
Looks similar right? Yes you are right, it is the auth_uri, only it has a few query parameters attached to it. I assume that the parameter definitions are clear since we discussed what each parameter here means in our last discussion. Let’s analyze these parameters and try to see from where they are coming:
- access_type: Coming from code line number 6.
- approval_prompt: Coming from code line number 7.
- client_id: Coming from the OAuth client JSON file provided in code line number 1.
- redirect_uri: This is a localhost URI since we are expecting the authorization code to be sent on local port 8888.
- response_type: It is a default value for web applications.
- scope: Coming from code line number 5.
Resource Owner grants the permission to Client
So far so good, we have the auth_uri printed out to the console. But now what?
Here comes the important distinction between the OAuth client and Service Account approaches: Resource Owner’s manual intervention. Yes, the resource owner has to manually click on the above auth_uri and login to its Google account and then grant the permission by clicking the “Allow” button on the OAuth 2.0 consent screen displayed.
Once the resource owner does this, the authorization code is generated (which represents the owner’s grant) and sent back to the redirect_uri.
Client requests connection to Authorization Server with Resource Owner’s grant
Still something feels missing right? I mean yes we have the authorization code now but we did not have to write any more code to interpret the code and exchange it for the access and refresh tokens right?
The SDK is taking care of that.
Since our client application is aware of the token_uri and also now has the authorization code, it can silently request an exchange without the resource owner’s knowledge. Cool !
Authorization Server responds with access and refresh tokens to the client
If you did not realize it yet, token_uri is nothing but the Authorization Server. It validates the authorization code and returns the access and refresh token corresponding to the resource owner.
What we don’t see is that this exchange takes place as part of this line:
The “Credential” object is the SDK class type that stores the access and refresh tokens for a user.
Client asks for permission to access resources from Resource Server
For us, the resources are emails and the resource server is Gmail. So let’s try to access emails from Gmail, now that we have the access token for the resource owner:
“Gmail” is the Gmail API SDK class type that knows how to interact with Gmail service. Please note that this line alone does not trigger any API network call to Gmail API as it does not specify the exact resource to be accessed or the exact operation on Gmail.
To specify the exact operation, we will use the corresponding Gmail API SDK client methods:
Get a list of emails from inbox:
Resource Server responds with the desired resource to Client
The above line of code will make the required network call to the resource server, which then validates the access token against the resource owner and if valid checks the expiry of the token. An invalid or expired access token will result in a 4xx HTTP response.
Although if the access token is valid and is not yet expired, the resource server responds with the requested resource.
Attention !
We are good with Google OAuth flow for operating on a user’s inbox now. But there are few important points that one should consider having while implementation:
- One could persist access token and refresh token with proper encryption into a DB. This is important because the application hitting google servers every time a user request comes, for fetching/refreshing access token, creates unnecessary overhead. So requesting a new access token only if it is expired could be an efficient approach.
- One should never place the OAuth client JSON file in the application server. The reason being, it will mostly be exposed to the public internet and hence will make your client’s data vulnerable. One could persist the file in a secure BLOB storage like AWS S3 and store the encrypted path to the S3 BLOB in the DB. This ensures the security of your client data.
- Remember, the refresh token becomes invalid as soon as the end client changes the password to its Gmail account.
- Placing a strict one-time retry policy for every Google API request made by your application, in case of intermittent network failures, could make your application a bit more robust.
- Building an application level caching mechanism for a user’s access token might help since it will reduce the number of calls to your DB.
What’s Next ?
Next up, will be an implementation of the Google Service Account approach for the same solution. While it requires some domain admin activities, we will focus more on what we as developers can do. Watch out for that !
