The new Google App Engine Standard Python 3.7 Runtime

This text was originally published on Feb 15, 2019 on my personal blog.

Today I’ll briefly discuss some of the differences between the new Google App Engine Standard Python Runtime versions: 2.7 and 3.7. Recently, I faced some trouble while upgrading some of my applications. So I thought I would compile this short journey along the way. In this post I’ll basically comment the key changes; in a follow up post I’ll exemplify the new environment by setting up a local development sandbox along with a basic project using Django e Google Datastore.

Why?

Python 2 has had a good run. Unfortunately its support will be dropped on January 1st, 2020 as stated on the PEPs 373 and 404. But fear not my fellow lover of the programming arts, we are excellent hands with Python 3 for at least the year 2024 (PEP 569). For that reason many influent libraries and frameworks are also following this trend. They are stopping the support on their Python 2 flavor and going full Python 3. This is no exception to the Google Cloud. Well, to be honest, there’s a lot of functionality still lacking support on the Python 3 counterpart. I don’t blame them; it’s impressive the amount of code and documentation they support. I assume they will take their time rolling this upgrade. Better safe than sorry.

The goal of this article is to expose the differences between working with Google App Engine’s Python 2.7 and 3.7 runtimes and finish with a practical example on how to setup a project using the new runtime along with Django and Google Datastore.

What is new?

Since December 14, 2018 the Python 3.7 runtime for Google App Engine Standard environment is GA. And I must say this was not just a package upgrade. The goal was to make your Python development feel more natural. They want to offer you a “idiomatic Python experience”, as they state in their documentation. Gone are the days where you were shackled by arbitrary counterintuitive practices to make your Python application be compliant with the App Engine sandbox. I’ll sum up the differences I felt more important, but you can read the full changes in here.

As mentioned above, the new runtime provides you more freedom and generality. There were times during my early development with the 2.7 runtime that I found myself trying to use some libraries within the sandbox only to find that they were incompatible (libraries not written in pure Python) or incorrect (some libraries in the Google Cloud Python SDK were made to be used within the sandbox and others in any other environment, such as a local desktop App). Furthermore, the only way to pack third party libraries that were not provided natively in the runtime we had felt really awkward. Anyway, all that is gone. Now if you need a new package in your App, just add it to a requirements.txt file just as you'd do in any other Python project. The listed packages will be installed when you deploy your App. That is so much better.

Now you can access the file system on /temp, spawn threads and use all Cloud Client libraries as in any other development environment . The way app.yaml works was extremely simplified. Many tags were deprecated and setting it up is just simpler. I know it seems that everything improved in the new runtime, and in a way it is true, but there is a caveat. Many of the eases and functionalities developers had when using App Engine APIs are gone because they are simply not supported. Here are some of the functionality that is gone:

• Blobstore
• Capabilities service
• NDB Datastore library
• Images service
• Images sandbox
• Request automatic logging
• Mail service
• Memcache Service
• Modules API
• Search service
• Users service

Truth be told, you can have all that functionality. The thing is that it is not a given anymore. In the past if you wanted a high abstraction Datastore wrapper with automatic managed caching, just import NDB Datastore library and use it. Now, if you want something like that, you need to code it (or find an open source project that already has done that for you). The point is that in the 2.7 runtime many things were easier but you had to follow the “App Engine’s way”, if you will. For example, if you used any of the App Engine APIs, you had to develop your App within dev_appserver. You couldn't just python main.py it (that is a joke, I meant using your preferred WSGI server in development). There would be all kinds of import error since that libraries only exist and work in a container inside App Engine inside Google's data center. So your code would not be portable and you would have to be fluent in "App Engine".

Now the approach is different, generality is key. dev_appserver is not the preferred way of local developing, but using an isolated virtualenv. Again, just as any other python App development. Finally, if you want to use other Google services use the generally available Cloud Client libraries.

Conclusion

If you like Python’s idiomatic approach, you’re probably enjoying this new App Engine Standard. I honestly liked them. Even with drawbacks of some lost functionality, I feel prioritizing generality is always better in the long run. It seems Google is giving developers more freedom to make things their way. Just careful, with power comes great responsibility. Making good Apps is not as easy as it was. Oh well, it is still way better than managing VMs and services by yourself; I’ll take it.