11 Things To Understand Before Migrating Your Python 2 GAE App To Python 3

If you’re reading this it might be because you have at least one web app running on the old GAE Standard Environment that supports only Python 2 (and of course Java, PHP and Go).

Meanwhile, the new generation of the GAE Standard Environment has arrived and it enables us to build web apps using Python 3. But switching from the old GAE environment to the new one is not just about changing a few lines of code in a Python 2 web app to make it compatible with Python 3.

Instead, the new GAE Standard Environment brought some significant changes that you should be aware of if you are planning to migrate your Python web app from the old to the new Standard Environment. Let’s take a look at them.

1. requirements.txt

Let’s start with good news. You won’t have to store all your third-party libraries in a special libs folder anymore!

Instead, you’ll define them in the requirements.txt file, just like in any regular Python project. The new GAE Standard environment will automatically detect this file and install the libraries.

It was about time, right?

2. Datastore vs. Firestore

Firestore is basically a new generation Datastore. So, should you migrate your data from Datastore to Firestore?

Actually, Google will do this for you in the background. The only difference is that if you used Datastore before, you would still need to use Firestore in a “Datastore mode” (or maybe better put: in the Datastore way).

You can read more about it here: Automatic Upgrade to Cloud Firestore.

So the good news is that you won’t have to do anything to migrate data from Datastore to Firestore.

But that doesn’t mean you won’t have to do anything when migrating your Python 2 GAE app code to Python 3. The main reason is that the ndb library is not (yet?) supported on Python 3.

Instead, you have to use a new library called google-cloud-datastore. This library helps you communicate with your Datastore (or with Firestore in the Datastore Mode). But the new library will require you to write the query code a bit differently than you were used to. See some examples here and here.

There are some gotchas to consider though. One is that you’ll have to put data in the Datastore as dictionaries (JSON) and you’ll also receive data back as dictionaries (not objects, as you did with the ndb library). So don't forget to add the dict_to_obj() method to your classes. ;)

index.yaml for creating composite indexes still works as before.

3. Running different Python instances

More good news. You can run your new Python 3 instance next to the Python 2 instance in the same Cloud project.

This means you don’t have to create a new Google Cloud project, but instead you can run Python 3 instance(s) in the existing project.

For example, you would be able to create a Python 3 instance with a special version name (like new or beta). This instance, of course, would have access to the same database as your Python 2 instance does. And you would be able to give your users a preview of the new/beta web app on the separate appspot.com subdomain (e.g.: https://new-dot-projectname.appspot.com).

Pro-tip: just make sure to use the --no-promote flag when you deploy your Python 3 instance:

gcloud app deploy --version new --no-promote

This means the new version will not become the default version.

4. webapp2

I personally really liked the webapp2 Python microframework, mainly because of its simplicity.

But it’s time to say goodbye.

Why? Because Google isn’t really doing anything to actively maintain it. They don’t even consider it as their official product, they “only host it” on their GitHub account and “can review any pull request” made by the community volunteers.

Even though webapp2 3.0.0b1 (beta) does support Python 3, I would advise against using it. Instead, you should rewrite your web app into Flask.

You’ll have to rewrite large parts of your web app anyway because many of the old Google Python 2 libraries (such as the ndb) won't work on the Python 3 environment. So why not change the whole framework along.

5. Cron jobs

This is the area where you won’t have to change anything (for now).

According to this documentation, you can use cron.yaml in a Python 3 GAE app in the same way, as you have used it in the Python 2 environment.

But be aware that Google created a new service called Cloud Scheduler (still in beta) which is set to replace App Engine Cron Jobs.

It is not yet clear if you’ll be able to use Cloud Scheduler via the cron.yaml file. But even if you won't, you'll be able to create scheduling jobs either visually at the GCP Console or using the gcloud command line tool. So no worries with this one.

6. Task Queue API (background tasks)

Here’s another GAE Standard library that cannot be used in the Python 3 environment: App Engine task queues.

Instead, you’ll have to use Cloud Tasks via the google-cloud-tasks Python 3 library. Beware: the service is still in beta.

7. Memcache

Memcache is not (yet) available for the Python 3 GAE Standard Environment. But according to Google, they are “actively working on making a Memcache solution available to the Python 3 runtime.”

If you’d want to migrate your GAE app from Python 2 to Python 3 today, you’d have to use Redis instead (see Cloud Memorystore).

8. Mail API

GAE allowed you to send emails via the Mail API. This worked for smaller web apps, but for anything serious, you had to turn to a professional third-party solution such as Sendgrid or Mailgun.

So, in case you did use the Mail API — guess what? Not supported on Python 3. Time to say goodbye. :)

9. Users API

This was one of the nicest features of the old GAE Standard Environment. Enabling Google login with just a few lines of code via the Users API.

The good old days are over and the Users API is not available on Python 3. Instead, you’ll have to use Firebase Authentication or, even better, the Requests OAuthlib.

10. dev_appserver.py

You can run your new Python 3 web app with the “old” dev_appserver.py on localhost. Which is kind of funny, because gcloud and dev_appserver.py still require Python 2 to run. So you still need both Python versions installed.

Also be aware that the Datastore emulator is moving out of dev_appserver.py and is already available as an extra feature in gcloud (although still in beta for now). See this link for more info.

11. No plans for deprecating the old Standard Environment

Finally, just to make you a little bit less stressed about everything: Google has no official plans to deprecate the old Standard Environment.

Even though Python 2 has the “end of life” in 2020, you will still be able to use it on Google App Engine. According to some Google insider, too much of Google’s infrastructure still relies on Python 2 so Google will take care of patching Python 2 if needed.

To conclude, some of the services a normal project would need on GAE are not available yet (Memcache) and others are still in beta (Scheduler, Tasks, etc.). So there’s no hurry with the migration. But you should start planning it.