Setting up a Python Google App Engine project in 2017: Quickstart

I’ve been working with Google App Engine since at least 2010, I think maybe since 2009, and for the last 5 years I’ve been working with it solidly in my role at xapiapps. During that time it’s gone from a small, simple system to being part of a large and complex offering, the Google Cloud Platform.

Recently, digging through the docs, I realised I’m using some pretty old techniques in some places, and I need to get up to speed with the latest SDKs and approaches. A good way to go is to start from the beginning.

In this article, I’ll go through setting up a new AppEngine project from scratch, using the latest SDK and such.

I’ve worked almost exclusively in Python on AppEngine, so I’ll stick to that.

—

First of all, always read the docs. For this article I’ll be using the Python Standard Environment. And hey, let’s follow the Quickstart guide.

It says to do these two things first:

install the Cloud SDK and create or use an existing Cloud Platform Console project

So let’s do these two things.

Install the Cloud SDK

This is the first bit where my knowledge has gotten old. There used to be a standalone AppEngine SDK, which I’m still using everywhere. I’d better try this newfangled Cloud SDK thing!

Ok.

I’m a Ubuntu user, so I’ll follow the Debian/Ubuntu instructions. Ooh look, it’s a package! The old SDK was a download that you had to extract, stick the files somewhere, bleh. So this is potentially great!

btw, if you need a Windows guide, there’s an excellent one here.

Following the instructions:

1: Create an environment variable for the correct distribution:
export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"

ok, I added that to my ~/.bashrc file. This is a dev machine, I always log in as me, so that’ll do. I restart my terminal, let’s see what it did:

xapiapps@tes(17:23:38):~$ env | grep CLOUD_SDK_REPO
CLOUD_SDK_REPO=cloud-sdk-trusty

Righto, I guess that’s important.

2: Add the Cloud SDK distribution URI as a package source:
echo "deb https://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

Seems legit. Done.

protip: this is always safe, just always do this no matter who tells you to :-)

3: Import the Google Cloud public key:
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Likewise, legit. Done.

4: Update and install the Cloud SDK:
sudo apt-get update && sudo apt-get install google-cloud-sdk

The magic.

Installing the package. Looks good!

Ok, I’ve got the basic package installed. But wait, it suggests some packages, and one is google-cloud-sdk-app-engine-python . Seems like something I’d want. Oh, and the next step says:

5: Optionally install any of these additional components:
  google-cloud-sdk-app-engine-python
  google-cloud-sdk-app-engine-java
  google-cloud-sdk-datastore-emulator
  google-cloud-sdk-pubsub-emulator
  google-cloud-sdk-bigtable-emulator
  kubectl

I use the datastore as well, but maybe I don’t need that yet? Ok, let’s install app-engine-python:

sudo apt-get install google-cloud-sdk-app-engine-python

Great. Gotta love Ubuntu, makes this kind of thing so easy (and congratulations to Google for finally using debian packages here).

There’s one last step, a magical incantation of some kind:

gcloud init

It does this:

And then I’m off authing in a browser window.

Hmm. I use more than one google account for appengine dev. I’ll pick the most likely one, and I’m going to assume there’s some way to add another one in the future.

I click on through the permissions Google asks for (sure Google, whatever), and get this page:

You are now authenticated with the Google Cloud SDK! | Cloud SDK Documentation | Google Cloud…

I’ve linked it here, because apparently you don’t need to actually complete auth to read it, and it seems to have some useful links and so forth. I’ll come back to it.

But wait, wait, I clicked on my terminal window, and it’s doing a thing! Ooh! Here it is:

Lucky I found that!

The whole thing looks complicated to me. Better look at the gcloud reference.

After some scrambling, I figured out I can manage authentication via “gcloud auth”.

So wait, to switch accounts I have to type “gcloud config set account emlynblahblabo@waitiforget.what.Im.doing.com”. wat.

Let’s try some help:

gcloud --help

My God, It’s Full of Stars

Ok, forget all that now. What was I trying to do? Oh yes, get the quickstart example going.

I feel like the sdk is installed, if not correctly configured. Let’s segue off to creating a Cloud Platform Console project, then we can come back and see what the quickstart guide wants us to do next.

Create a Cloud Platform Console project

I’ve got plenty of existing cloud platform projects, but I’m going to create a new one for this next step.

This is pretty easy. Just start here:

https://console.cloud.google.com/project

Then create a project.

I’m creating the project “emlyn-experiments”. Wait for the system to buzz-whirr-click, then I get this:

I’ve created this project to reuse for doing experiments on GCP and appengine. Billing is turned on (and is supported by xapiapps.com, thanks chaps!).

I chose a unique name, so my project id is the same as what I chose. If you choose a non-unique name (like “Fred”), google will disambiguate it for you (like “Fred-16364950”). That’ll be shown as “Project ID: Fred-16364950” on the dashboard. You need to know that name, you’ll use the project id with gcloud when developing.

So far I have gcloud installed, and I have a project-id, but I haven’t tied them together. Hopefully that doesn’t matter? Let’s just get back to the quickstart.

Get the sample project

Back on the quickstart page, it says to get the sample project by cloning a git repo:

git clone https://github.com/GoogleCloudPlatform/python-docs-samples

Before I do that, I’ll cd to a folder to work in. I’m putting it in the folder ~/dev/evcws, so I’ve cloned the repo in there.

Now they say to cd to a subfolder where the helloworld app is:

cd python-docs-samples/appengine/standard/hello_world 

Done. So why did I have to do that? So I can run the application. Cool.

Run the app locally

In the quickstart they say “Test the application”. What this really means is that I’m about to run it locally. This uses a local machine version of App Engine, called dev_appserver.py, which lets me run code locally as if it were running on App Engine proper (with some serious caveats).

dev_appserver.py app.yaml

This command starts dev_appserver, using the file app.yaml in python-docs-samples/appengine/standard/hello_world as the starting point of the app. app.yaml is like a manifest for the application, telling App Engine how to interpret the gob of source code I’ve presented it with.

I then see this:

The app is running at localhost:8080 on my machine, and there’s an admin console at localhost:8000.

There are command line arguments to change these ports if you need to, have a look at the Python Development Server reference.

Let’s look at localhost:8080:

The Helloworld app running locally

Nice. How about the admin console?

The admin console is really useful, have a poke around.

Ok, that’s great. I stop dev_appserver.py by pressing CTRL-C:

That’s a clean shutdown, just what you want.

Running the app locally is great. Super useful actually; you can edit code and dev_appserver.py updates in real time, really good stuff.

However, what we really want is to get this thing running on App Engine proper. Let’s get that happening next. The quickstart guide says we’ll be “deploying”.

Deploy your app

The quickstart says to stay in the same directory (the one with app.yaml) and run the following command:

gcloud app deploy

I’m going to need a switches though, to send this app to the right place.

Authentication is set up already, but gcloud doesn’t know which Project ID to use. So I use the --project switch, with my Project ID.

gcloud app deploy --project emlyn-experiments

Oh man, that sounds scary!

I think the “app” thing just isn’t a problem at all. I’m guessing this is an App Engine app? Not sure.

In the past, I’ve never been able to choose a region. Now we can choose. Cool.

Given the recent election in the US, anywhere US hosted is out if at all possible; let’s keep our data away from tiny, grasping hands. And the EU is all neurotic about data and such, so let’s just spare them any more anxiety. I’m in the asia-pacific region, and as a region it’s a kind of lawless, do what you like kind of place as far as data goes, which is fabulous. Ok, asia-northeast1, I choose you!

Hmm, lots of tech-the-tech. But it looks like what I want to do. Ok.

Here’s uploading in action. Pretty quick!

Now, it says

Deploying to URL: [https://emlyn-experiments.appspot.com]

That looks pretty good. Let’s check that url out:

Oh, a win!

—

How’s it work?

Let’s poke around in the sample app.

There’s not much in here. Here’s app.yaml:

runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
  script: main.app

What this says is the following:

• runtime: python27 — Use the python27 runtime. ie: we’re programming in Python 2.7.
• api_version: 1 — Something to do with how appengine interprets app.yaml and the rest of the application. This has always been 1 since I started using appengine. Just set this to 1.
• threadsafe: true — This means your instances, think of them as single virtual machines running python in a sandbox (I’ll talk more about them in another post) can handle multiple requests at the same time, running them in multiple threads. It means dramatically lowers the cost of frontend instances in your application (you pay the same for an instance servicing one request or multiple) and generally if you don’t do anything weird, your code is actually threadsafe. Weird things would include using global memory in a non-threadsafe way, and using non-threadsafe libraries. You can set this to true and really ignore it, but keep in mind the weird things.
• handlers: — This is a list of routes and how they are handled in your application.
• - url: /.* — This describes a route, using a regex. It is anything that starts with a slash, then zero or more characters. ie: this is the everything route.
• script: main.app — This is super weird notation, python module stuff. It pertains to the route above (everything) and says you should handle everything for that route by passing it to the global name app declared in the module main.

So this app.yaml says we are using Python 2.7, we’re threadsafe, and all requests should go to the module main, to the global name app.

Let’s look in main.py (ie: the module main):

import webapp2
class MainPage(webapp2.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.write('Hello, World!')
app = webapp2.WSGIApplication([
    ('/', MainPage),
], debug=True)

First it imports webapp2, which is made available by AppEngine itself, and provides the framework for Python 2.7 standard runtime apps.

At the bottom of the file, you see the global name app being assigned the a WSGIApplication instance, with a route defined: ‘/’ maps to a class MainPage.

Then in the middle of the file, we see the MainPage class’s definition. It descends from webapp2.RequestHandler (which it needs to, to be used in webapp2.WSGIApplication).

It defines a method get(), which it uses to handle GET requests. If it had a post() method, it’d handle POST, but it doesn’t, so it doesn’t, if you get my drift.

And you can see in the body of get() the code for returning ‘Hello, World!’ which we see when we go to url of the app.

Simple!

main.pyc is the compiled version of main.py (probably wasn’t there until I ran the app), but what’s this main_test.py?

import webtest
import main
def test_get():
    app = webtest.TestApp(main.app)
    response = app.get('/')
    assert response.status_int == 200
    assert response.body == 'Hello, World!'

I’ve never seen this stuff before. webtest looks like some kind of unit test framework for Python WSGI apps, for hosting and testing the WSGIApplication. That’s pretty amazing. I’m currently using from google.appengine.ext.testbed in unittests to mock app engine, but I didn’t know about actually calling my request handlers in unit tests; that’s potentially the bee’s knees! Needs more investigation, but that’s for another post.

—

This is about as basic as an App Engine app gets. I’m interested in things a tad more complex, but step by step ey? Thanks for reading, if you got this far.