The Matrix - OAuth 2.0, Service Accounts, Google Drive and Node.js

What if you want to upload a file to your Google Drive? Simple. Just use the upload button and it’s there. But, what if you were a web application? Now things get complicated. And there’s no such thing as your Google Drive. For you every Drive becomes the user’s Google Drive.

OAuth 2.0

To do this, you have to create a Google API Console Project and enable the Drive API on it. Then you’re required to setup, what is called, the 3-legged OAuth. Imagine 3 persons, the user, Google and your-app, standing, making an agreement. Under the agreement, user allows your application to access his data using Google APIs.

In practise, this will involve user getting taken to a consent screen where the user authenticates with Google and gives the consent. In our case that will mean access to the user’s Google Drive.

This is standard OAuth procedure. The key things to take from this process are:

1. User consent is mandatory - And it should be. Since an application is trying to access personal data.
2. Consents need user interface - That can be the browser. Even if your application is pure script, without any UI, Google will create a URL which must be pasted in browser from the terminal. You can try their Sheets API quickstart for an example.
3. Tokens must be stored - The final product of the authentication process will be a token(a long string). Application sends this token along with API requests. So these must be stored somewhere. Or the user goes through the consent process again. Mostly, storage will be a database. The Sheets API example, which is written for command line, uses local file system as storage.

Oh, and the tokens expire periodically. They must be renewed. Mostly these things are handled by a library. For example, Google API Client Library. But, as said before, some form of storage is required.

So far in order to access users’s Google Drive your application needs user consent, a UI in an environment like the browser and some form of storage. Lot of work but seems reasonable.

But how to do it in an automated manner? What if the application runs on a VM? Then there’s no user to give consent. It’s just machines. If you want to know why I am obsessed about automation and remote machines; you can ask these folks, who are building a social reading platform.

Service Accounts

Service account are used when applications or machines want to authenticate with each other.

To use them you have to create one in the same Console Project, where you’re enabling the Drive API. Follow the Google’s documentation for creating one. Here are the important bits:

1. You can pick any name for your service account.
2. For role just choose project owner.
3. The only thing you want is the service account key which will have the client_email and private_key . This key is downloaded automatically as a json file. Don’t loose it.

You can think of service accounts like normal Google accounts but belonging to bots. It has a unique email address. Here’s the one for the service account I created for this post:

agent-smith@in-your-service.iam.gserviceaccount.com

Where agent-smith is the name for my service account and in-your-service is the project name.

Service accounts use 2-legged OAuth to authenticate.

Your application calls Google APIs on behalf of the service account, so users aren’t directly involved

Hence no user consent. Primarily, because APIs with which they are meant to be used, don’t access any user data. For example, if your node app wants to use the Cloud Datastore as database or Cloud Storage to store files; a service account will be used to authenticate. You can see these APIs are rather linked to the console project than with the user.

But Google Drive is different. It belongs to the user. And service accounts can’t be used to access user data. Or can they?

Service Accounts and Google Drive

No more talking. We will write a simple node app which will upload a file to Google Drive. Our service account, agent-smith , will help us send neo.txt to Drive aka The Matrix. We’re the baddies here.

The code can be found on GitHub. Here’s the directory structure:

/service-accounts
|-- neo.txt
|-- index.js
|-- agent-smith.json

Authenticating Using Service Account

agent-smith.json is the downloaded service account key. It, basically, represents the service account. Now let’s use it to make an authorized request to list all files on the Drive. Put this code in index.js and run it using node index.js from the terminal.

Authenticate using service accounts

Terminal output for listing files on Drive

The important step in the above code is creating jwtClient object where 4'th argument to it is an array of scopes. Scopes control the permissions our app will have on the Drive. We’re giving it full read, write access.

Uploading File to Google Drive

And now to upload the neo.txt to the Drive; you can comment out the entire list command above, replace it with this code and run it.

Uploading file

Terminal output for uploading file to Google Drive

It looks like we’re done but there’s a small problem. You won’t be able to find neo.txt on the Drive. But even before that, whose Drive are we even talking about? It’s not the one for the Google Account, which you are using to create the API Console Project to use the Drive API.

Running the list files code again still shows 2 files for me including neo.txt .

Listing files on Drive after uploading neo.txt

And what’s this Getting Started.pdf ? I started with a completely blank Drive. You might have that file but can try deleting it and listing the files again in terminal to see it’s still getting logged.

My Google Drive was empty before running any code

What going on over here? And, more importantly, where’s neo?

I avoided this point on purpose so that we could see the problem first. When authenticating through OAuth with user consent, our application makes request on the user’s behalf. So the when application runs code to list files in the user’s Google Drive, it is as if the user is performing those actions. That’s the whole purpose of authentication.

But it’s different with service accounts. There’s no user. But we still make request on service accounts’s behalf. Does this mean that we are accessing service account’s Google Drive? More importantly, does the service account has it’s own Google Drive?!

The answer is yes. It seems agent-smith is more real than we thought.

Granting Drive Access to Service Accounts

Not only service accounts have their own email but they also have their own Google Drive. I could not find it in any docs but it seems that Drive space is shared with the parent account, but there’s no GUI to access it. The only way to access agent-smith's Drive is through the API. Pure evil.

‘Illusions, Mr. Anderson. Vagaries of perception.’

Now we’re back to square one. How do we give the access to our Google Drive to the service account? Or any Drive for that matter. The answer is simple. Share it.

I will create a folder called The Matrix on my Google Drive (it can be on anybody’s Drive). Then, share that folder with service account using its email which could be found in the downloaded json.

Sharing folder with the service account

Now running the list command again give will also list The Matrix .

Listing the shared folder

Sending neo to The Matrix

All that is left is to update our code where we upload the file. We need to add only one line of code to the fileMetadata object:

// parents is an array containing folderId strings
const fileMetadata = {
  name: 'neo.txt',
  parents: [folderId]
};

You can get the folderId simply by opening the folder and copying it from the browser’s address bar.

Folder ID

Of course we can get the folderId programatically. But this post is already too long. Now with file upload code all set, run the script again which sends neo to the The Matrix . The world is doomed. We won.