Google OAuth credential: going deeper, the hard way

Security on Google Cloud is paramount but it’s strangely an unpopular topic. Actually, the security is often a boring topic! To fill the gap, I wrote articles about the 2 limits of IAM services and about workarounds and new use cases offer by Service Account Credential API. In both cases, my main concern was still the same: to avoid the users to download service account key files to improve the security.

However, the latest blog post on accessing Drive API of Gabe Weiss uses service account key files. That’s why I reached him out and the discussion was very interesting.

Gabe use case

In his blog post, Gabe shows how to access to a Google Drive folder. This folder could be confidential or, at least, you don’t want that any developers access it.
Indeed, if you grant the user account on the folder, the access is possible by API and also through a browser, which is a problem and that you don’t want.

So that, the solution is to grant the access autorisation to a third party, not the user account, but to a service account. And so, to use this account, you need to have the credential on your side and to use them

The service account key file!

And YES, the use case is valid, and can be duplicated for any Google Workspace products: access to a Google Doc, Sheet, Slide,..

Ok, but in my ideal world, you have a Google Driver folder for dev/test, and another one, the confidential one, for production. And this for 2 reasons

1. If the user is a bad actor and wants to read the files inside the folder, they can access them by APIs. At the end, the issue is the same: the user has access to the folder content, it’s just less easy to download content!
2. Worse, if the user uses the service account to access to the folder files, you only see the service account email in the logs, not the user credential. The bad actor is hidden by design!

That’s why, I would like to show you how to achieve the same access to a Google Drive folder but with a user credential.
And this way is absolutely no easy!!

Code sample update

Before going deeper, I would like to update the code proposed by Gabe to make it more “portable”, I mean to rely on the ADC (Application default credential). This solution lets the Google Auth library pick the credential available on the runtime environment, and thus, you don’t have to update your code according with the environment.

The credential recovery is performed in this order

1. Check in the GOOGLE_APPLICATION_CREDENTIALS environment variable exist. If so, use it (and should reference a credential file, such as a service account key file).
   This mode is done to override the 2 others and for no GCP environment
   If not, go next.
2. Check if the metadata server exists on the environment. This feature allows to use the service account loaded in the Google Cloud products.
   This mode is done for GCP runtime environment
   If not, go next
3. Look for user credential file created on the local environment, in “well-known location”. This file is generated with the command gcloud auth application-default login
   This mode is done for your local development environment

I split the code sample in 2 steps

1. Firstly, set up the environment with the GOOGLE_APPLICATION_CREDENTIALS environment variable

export GOOGLE_APPLICATION_CREDENTIALS=</path/to/>drive_icons_sa.json

2. Then the code, working on your workstation and on Google Cloud products’ service account in the same way.

from googleapiclient.discovery import build
drive_service = build('drive', 'v3')

Yes, only 2 lines!! Let the library work for you!
You can add the children example as provided in the Gabe article and then print the result to valide the correct work of the script.

children = drive_service.files().list(q=f"mimeType = 'application/vnd.google-apps.folder' and '{marketing_icon_folder_id}' in parents", fields='files(name, id)').execute()
print(children)

Test and validate that works as expected.

The long journey to use user credential

As seen before, the first things to do, is to unset the environment variable GOOGLE_APPLICATION_CREDENTIALS.

unset GOOGLE_APPLICATION_CREDENTIALS

And to create a user credential file in the “well-known location”

gcloud auth application-default login

Finalize the process and test the python script.
You got an error because your user credential doesn’t have the correct scope.

Insufficient Permission: Request had insufficient authentication scopes.

I will save your time, don’t try the Gabe code where he set the scopes explicitly on the credential: it doesn’t work on user credential.

The trick here is to scope the user credential when you generate the credential token.

gcloud auth application-default login \
  --scopes='https://www.googleapis.com/auth/drive'

If you use this credential to access to Google Cloud resources, add the google cloud scope, like this

scopes='https://www.googleapis.com/auth/drive','https://www.googleapis.com/auth/cloud-platform'

You can add all the scopes that you need: Sheet, Docs,…

So, now, have another try to the python code.
And FAIL again, with a strange error:

Access Not Configured. Drive API has not been used in project 764086051850 before or it is disabled. Enable it ....

And the long way start here….

Deep dive in the Authentications

What’s this project number 764086051850?
When you follow the provided link, it’s forbidden!
You can go over all your project, you won’t find it!

You use a project that is not yours, you need to activate Drive API on an unreachable project. What’s the fuck???

If you have a closer look to your user credential in your “well-known location”

#Linux
cat ~/.config/gcloud/application_default_credentials.json
#Windows powershell
cat $env:APPDATA/gcloud/application_default_credentials.json

You can see this client_id

764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com

So, if you know the OAuth2 authentication flows, one required to present a client_id and a client_secret on the authorization server. From there, the platform authenticate the client (here the gcloud SDK) that requests the authorization and continue the process.
You select your account, validate the authorization, and then get an authorization token.
The client application, here gcloud SDK, get the authorization token and validate it. As answer, it receive an access_token and a refresh_token.

The client_id, client_secret and the refresh_token are saved in your “well-known location” file

The problem is: Because it’s a client_id from a Google Cloud internal project, it’s this project which is referenced in the request.
And, of course, you don’t have access to it to enable the Drive API

So now, how to use a client_id from your project?

The gcloud CLI allow you to do this with the param --cliend-id-file

But how to generate a client ID file?

For this, you can go to the console in the APIs & Services, click on Credentials

Then Create Credential of type OAuth client ID

Up to here, it’s quite obvious. The next is far lesser.

You have to choose an application type. Good luck, pick one!! If you don’t pick the correct one you will have an issue like this

Only client IDs of type 'installed' are allowed, but encountered type '...'

What’s the application type installed in the list?

So, choose Desktop app!

Name your key and create it.

Eventually, you can download your key by clicking on the download icon on the right hand side.

Save the file. Now you have downloaded a secret!!! But it’s only usable to authenticate a user, not to access to resources as a service account key file.

You have all the pieces! Time to generate your correctly scoped credential on your project!

gcloud auth application-default login \
  --scopes='https://www.googleapis.com/auth/drive'
  --client-id-file=/path/to/client-id-file.json

You will have security warning because the client is officially reconized by the authorization server. Continue, and validate all the steps.

And test again the python script.
Don’t forget to share the target folder with your current user email!

BOOM, that works!!

Choose the right flow

Now you have it!! But my feeling is mitigate.

• On one hand you have a quick and easy path with a service account key file. It’s not as clean as I can expect but the caveat is easy to explain to the developers for they take care of this secret.
• On the other hand, you have a more secure mechanism, without secret shared and involved, but quite impossible to achieve alone by the developers.

The security is paramount, but if it becomes a challenge to achieve easy tasks, it will be bypassed by users to meet their deadlines and expectations.

The tradeoff isn’t so simple and a better technical security solution won’t always bring a better overall security. Choose wisely!