Google OAuth 2.0 Access Token and Refresh Token Explained

User authentication at Google can be a bit confusing, especially the difference between the Refresh Token and the Access Token. When building an oAuth2 integration developers run into three common problems:

1. The Refresh Token is mysteriously only returned sometimes.
2. The Access Token expires randomly.
3. The Access Token is not being generated.
4. The workflow job API Calls are surprisingly slow.

Man pointing to complex map representing oAuth2 complexity.

Different Terms

For the rest of the article, the following two terms are used often:

Consent Screen: The place where the user approves credentials to be used for future API calls. This is usually referred to as the User Flow in oAuth2 credentials libraries.

API Call: A request made to a Google API during the execution of a job. Not to be confused with the Consent Screen.

Access Token: The thing used in the job API Call to authenticate.

Refresh Token: The thing used to get a new Access Token when the Access Token expires ( it does so every 6 hours or less ).

Different Use Cases

Short Jobs

For the purpose of short jobs that complete within Access Token time, the Refresh Token can be completely ignored because:

1. User goes to Consent Screen and approves scopes.
2. Redirected back to local UI or tool with both Access Token and Refresh Token.
3. Ignore Refresh Token because we plan to finish the API Calls fast.
4. Run API Calls with Access Token and complete before it expires.
5. Next time user wants to execute task, go through Consent Screen step again and start all over.

Long Jobs

For the purpose of long running jobs or jobs that may run at a later time the Refresh Token is critical.

1. User goes to Consent Screen and approves scopes.
2. Redirected back to local script with both Access Token and Refresh Token.
3. Save the Refresh Token and optionally save the Access Token. The Access Token is optional because it can now be re-generated using the Refresh Token anytime.
4. For a job starting immediately, use the Refresh Token returned with the oAuth2 call to the Consent Screen.
   OR
   For jobs starting later, the Access Token would first be loaded using the Refresh Token. Fortunately most oAuth2 libraries will do this automatically if you pass an expired Access Token with a valid Refresh Token. No Consent Screen is shown to the user but a new Access Token is fetched with the scopes locked to what the user originally approved.
5. Run API Calls with Access Token.
6. If the Access Token expires while running the job, use the Refresh Token to get a new Access Token, again this is done automatically by the oAuth library you are using by simply supplying the Refresh Token.
7. Keep running the API Calls with the new Access Token.

Problem #1: Refresh Token Missing

How the initial user Consent Screen interaction is set up matters. The oAuth call does not always return a Refresh Token because the developers thought sometimes you may not need one so skip sending an unnecessary high security thing. There are two ways to get the Refresh Token via oAuth2:

First Time oAuth2

Where the user has NEVER logged in before, will send the Refresh Token and the Access Token. After that, every call to oAuth2 will return only the Access Token. The application is expected to catch and store it the first time.

If you are the user, you will see the Consent Screen only once, after that if the application runs the authentication flow again the Consent Screen will automatically redirect back to the application without the Refresh Token and only the Access Token. The user is saved the trouble of clicking OK.

1. Your application should check if the oAuth2 response contains the Refresh Token.
2. If yes, this is the first login, store the Refresh Token for long term use.
3. If no, the Refresh Token is already stored, load that and use it in whatever credentials library you use.

If you are a developer and make two or more calls to oAuth2 but don’t save your Refresh Token from the first time, you will have to:

1. Delete the application from your profile to make it look like you’re logging in the first time again.
2. Force the Refresh Token to come back every oAuth…

Forced oAuth

The Consent Screen takes some parameters to force the user to consent and to return the Refresh Token. Example UI and Library code.

1. Add prompt=consent&access_type=offline to force the Consent Screen user input and the return of a Refresh Token every time. Probably best for development but not ideal for UX.
2. Add prompt=&access_type=offline to have Google determine if the Refresh Token needs to be returned based on user state. This is where the application should always be ready to update the Refresh Token in its storage if one comes back.

Problem #2: Access Token Expires Randomly

This usually happens in a distributed or multi process application. Everything is running fine, and then randomly every few hours an API Call fails with permission denied.

There are only 100 Refresh Tokens allowed in rotation at any time. So after 100 Refresh Token refreshes, the tokens start becoming invalid, and so do the associated Access Tokens. Other jobs in a distributed system will start triggering API Call permission errors randomly. Not an issue for single process, thread, machine execution but most systems are distributed so eventually problems occur.

Solve this by storing the Refresh Token in a shared cache. Note that when the user triggers another authentication or new scope, the Refresh Token is renewed. Up to 100 user triggered refreshes can occur before in progress jobs start failing.

Problem #3: Access Token Not Generated

When the Refresh Token becomes stale, it will no longer generate Access Tokens. The application may require additional logic to detect a stale Refresh Token and force and Consent Screen event to get a new one. A stale Refresh Token can happen for two reasons:

1. The user removed application access via the Google Workspace Account panel. The application will start seeing permission errors and needs to ask the user for another oAuth.
2. Too many Refresh Tokens have been generated, causing the limit of 100 to expire in use Refresh Tokens. The application needs to slow down the generation of new Refresh Tokens.

Most commonly developers encounter this during testing. A stale Refresh Token copy on the local machine is the culprit. Just delete it and re-generate a new Refresh Token.

Problem #4: API Calls Are Surprisingly Slow

The main reason for slow API Calls is the application tries to refresh the Access Token every API Call to keep things simple. Although this works, it is painfully slow and even throttled by Google.

There is a latency to requesting an Access Token. Usually the oAuth library handles this so it’s not explicit. The automatic calls do take enough time to make rapid API Calls slow down considerably if the Access Token is fetched for every API Call.

Solve this by using the Refresh Token for its intended purpose, to refresh the Access Token when it expires. It’s usually as easy as adding the Refresh Token as a parameter to whatever oAuth library you are using.

Conclusion

Please defer any advice to the official Google oAuth2 Documentation. Hopefully this saves you some time!