goa v2 on Google App Engine

A walk-through

Google App Engine (GAE) is a platform provided by the Google Cloud Platform (GCP) for building web applications. It allows developers to build and deploy their applications quickly without having to managing the instances underneath. We can even design microservices architecture on GAE as it is possible to deploy multiple services within a GAE project.

Obviously there are a lot of ways to write microservices application with Go. One of the approach that I find interesting is the goa framework which emphasizes on the design of the services. It lets us focus on writing and testing the codes that matter and generates the boring stuffs for us.

The purpose of this post is to show how to get started in building an microservices application on GAE with goa v2. Note that the current release of v2 as of this writing is still v2.0.0-wip. Example of goa v1 on GAE can be found in here. As the post will not be able to cover every details of goa v2 during the walk-through, please refer to the following documentations for more details about the framework:

1. The teaser.
2. goa v2’s docs.

(This is a detail walk-through. The codes are available in this repo.)

The Go gopher was designed by Renee French. (http://reneefrench.blogspot.com/)

Getting started

Before we begin, make sure that Google App Engine Go Standard Environment are set up, i.e. complete (only) the “Before you begin” section of the Quickstart by GCP. Then, create a new directory inside your GOPATH as you would do for any Go project, e.g. if the import path of the Go project is

$ MYIMPORTPATH="github.com/shihanng/gaegoasample"

then,

$ PROJROOT="$(go env GOPATH)/src/$MYIMPORTPATH"
$ mkdir -p "$PROJROOT"
$ cd "$PROJROOT"

Designs

The next step is to design the APIs with goa. For this demonstration our default service servicea holds an endpoint /api/info. It will return the basic information about itself as data Info and 200 OK when receiving a GET request. The service has its directory located within the $PROJROOT/scv directory and within that directory there is a design/ directory which contains the API design design.go file.

$ install -Dv /dev/null "$PROJROOT/svc/servicea/design/design.go"

svc/servicea/design/design.go

The additional import path is an alternative solution to avoid the issue describe in here.

import _ “goa.design/goa/codegen/generator”

This is what our project directory should look like up to this point:

.
└── svc
    └── servicea
        └── design
            └── design.go

Code Generation

The next step is to generate the Go codes for the service which we will be using later during the implementation.

$ goa gen "$MYIMPORTPATH/svc/servicea/design" \
  -o "$PROJROOT/svc/servicea"

Note that the first argument of the goa gen command is the import path of the design package and we are using the -o option to specify the location of the generated files. Files generated by goa gen should not be modified manually. When API changes become necessary, we should make changes in the design package and regenerate the codes with goa gen again.

Implementations

The amount of generated codes might seems overwhelming. For the actual implementation, we can use the goa example command to generate the scaffolding to assist us.

$ goa example "$MYIMPORTPATH/svc/servicea/design" \
  -o "$PROJROOT/svc/servicea"

Among the four files that are generated,

svc/servicea/api.go
svc/servicea/auth.go
svc/servicea/cmd/servicea_cli/main.go
svc/servicea/cmd/servicea_svc/main.go

we are only interested in two for this tutorial: api.go and servicea_svc/main.go. Here are the changes that we will make on api.go.

The changes in Info(ctx context.Context) method shows that the /api/info endpoint will return the App ID, Module Name (Service Name), and Version ID of servicea that it gets using the google.golang.org/appengine library.

Integrating with Google App Engine

Now that we have the actual implementation, how do we integrate our service with GAE? What we’ve create so far is the library of the service svc/servicea/. We need the driver for the actual service and it is recommended to put that in a separate directory.

$ mkdir "$PROJROOT/servicea"

In that directory we have an ordinary app.yaml file like any other GAE project.

servicea/app.yaml

We also have a driver of our default service: main.go which is based on the example provided by google.golang.org/appengine where the "github.com/shihanng/gaegoasample/svc/servicea/cmd/servicea_svc”

package are expected to register HTTP handlers in their init functions.

Recall that the package is generated via the goa example above and it contains only one file: main.go (not an importable package). Therefore we will make some changes on the package:

• Make it importable by renaming the main.go file to servicea_svc.go and change the package name to servicea_svc.

$ mv "$PROJROOT/svc/servicea/cmd/servicea_svc/main.go" \
  "$PROJROOT/svc/servicea/cmd/servicea_svc/servicea_svc.go

• Replace the main function with the init function and remove all the generated CLI flags.
• Instead of starting the HTTP server we will just register the handler.
• Attach a simple middleware appEngineContext to the handler to include the App Engine context. Without it, we will get “panic: not an App Engine context” error when attempting to call an appengine’s function e.g. appengine.AppID(ctx).

Deploy to GCP

Before deployment, make sure that

• the GOPATH is set, e.g. export GOPATH="$(go env GOPATH)",
• the dependencies are properly vendored and can be found in $PROJROOT/vendor/.

For deployment use goapp deploy instead of gcloud app deploy as the latter does not work well with the vendored dependencies.

$ goapp deploy -application <YOUR-APP-ID> servicea/app.yaml

To confirm that things went well, visit the /api/info endpoint of the deployed service and we should be able to see something like

{"id":"<YOUR-APP-ID>","service_name":"default","version":"1.410480874412145332"}

We can see that the resulting structure of the project directories (with some files not being displayed for the sake of brevity) follows the best practices recommended by GCP:

.
├── servicea
│   ├── app.yaml
│   └── main.go
├── svc
│   └── servicea
│       ├── api.go
│       ├── auth.go
│       ├── cmd
│       │   ├── servicea_cli/...
│       │   └── servicea_svc
│       │       └── servicea_svc.go
│       ├── design
│       │   └── design.go
│       └── gen
│           ├── api/...
│           └── http/...
└── vendor/...

What we’ve done might seems too much for a service that has only one endpoint. However, API services in practice usually have more endpoints and that is where goa shines: it not only helps us build the APIs, but also generates the API client, CLI tool, documentations, etc. Adding new endpoint to the example above only requires us to modify the design design.go and the actual implementation api.go. We will leave that as an exercise for the reader.