Go, Cloud Endpoints and App Engine
Part 1
Preamble: I have been hearing a lot about Google Cloud Endpoints, App Engine and their interface to Go. As a person new to Go, I could not find any introductory tutorial on the same.
These study notes is an attempt to write about it as I explore and experiment, in the hope that a new Go developer can understand and get up-to-speed with it in no time.
I have made extensive references to articles that I did find on the web.
Thanks to:
- Romin Irani’s (GDE: Google Cloud Platform) excellent Tutorial Series on Google Cloud Endpoints for Java.
- Google’s overview of Cloud Endpoints.
- Google’s tutorial on Google App Engine.
APIs
Public APIs are now considered to be the most effective mechanism to integrate two applications. Typically the APIs are hosted in a server side application and exposed to the clients to consume. The clients could be:
- other server-side applications,
- native mobile applications or
- even browser applications (mobile or desktop).
When we speak about a Public API, few things are common to them:
- They are available over a popular and well-defined protocol, usually HTTP.
- It uses JSON or XML as the data format.
The above are not absolutely binding, but the majority of the Public APIs have that in common.
The REST protocol makes use of various HTTP verbs in an effective manner to perform basic operations:
- List (or Get) All Records
- Get One or more Records based on some search criteria or ID(s)
- Create a Record
- Update a Record
- Delete a Record
One key point about Web Services is that it provides a great layer of separation between the client and the server in terms of the technology used. Client applications are completely agnostic to the programming language/environment that you are using on the Server side. HTTP is the common language that binds the two together and as a result, you could write your Server side API implementation in Go but still have the ability to invoke the same HTTP API via a Go client, iOS client or even a Web client.
Google Cloud Endpoints
Google Cloud Endpoints, a solution from Google, consists of tools, libraries and capabilities that allow you to generate APIs and client libraries from a Google App Engine application, referred to as an API backend (i.e. it helps you create a Public API for your App Engine application), to simplify client access to data from other applications. Endpoints make it easier to create a web backend for web clients and mobile clients such as Android or Apple’s iOS.
The API backend is an App Engine app, the developer can use all of the services and features available in App Engine, such as Datastore, Google Cloud Storage, Mail, Url Fetch, Task Queues, and so forth. And finally, by using App Engine for the backend, developers are freed from system admin work, load balancing, scaling, and server maintenance.
The diagram below shows the Cloud Endpoints solution (taken from the official documentation):
The point to note is that you need to have an App Engine application. Apart from that, how you store your data or retrieve your data is completely up to you. You could opt to work with the data store to meet your API functionality or simply talk to external services too to aggregate and present the response.
Google App Engine
Google App Engine is different from most other cloud systems because it is neither IaaS (Infrastructure-as-a-Service, e.g., Amazon EC2) nor SaaS (Software-as-a-Service, e.g., Salesforce). It is something in-between — PaaS (Platform-as-a-Service).
Instead of a fixed application (SaaS) or raw hardware (IaaS), App Engine manages your infrastructure for users. Furthermore, it provides a development platform… users get to create apps, not use the ones provided by the cloud vendor, and it leverages the infrastructure as a hosting platform.
Google App Engine lets you run web applications on Google’s infrastructure. With App Engine, there are no servers to maintain: You just upload your application, and it is ready to serve your users. App Engine costs nothing to get started. All applications can use up to 1 GB of storage and enough CPU and bandwidth to support an efficient app serving around 5 million page views a month, absolutely free.
Creating an App Engine application is easy, and only takes a few minutes. And it is free to start: upload your app and share it with users right away, at no charge and with no commitment required.
Sandbox
Developers would not be interested in letting other applications/users get any kind of access to their application code or data. To ensure this, all App Engine applications run in a restricted environment known as a sandbox.
Warning: Because of the sandbox, applications can’t execute certain actions. These include: open a local file for writing, open a socket connection, and make operating system calls.
Services
The App Engine team has created a set of higher-level APIs/services for developers to use.
- Want your app to send and receive e-mail or instant messages? That’s what the e-mail and XMPP APIs are for!
- Want to reach out to other web applications? Use the URLfetch service!
- Need Memcache? Google has a global Memcache API.
- Need a database? Google provides both its traditional NoSQL scalable datastore and access to the relational MySQL-compatible Google Cloud SQL service.
The list of all the services that are available to users changes quite often as new APIs are created.
The Administration Console
The Google App Engine Administration Console gives you complete access to the public version of your application. Access the Console by visiting this link in your web browser. Google recommends that you use the Google Developers Console instead. The Developers Console supports all the Cloud Platform products, including App Engine, as well as other Google developer APIs.
Applications (web and non-web)
While many applications running on Google App Engine are web-based apps, they are certainly not limited to those. App Engine is also a popular backend system for mobile apps. When developing such apps, it’s much safer to store data in a distributed manner and not solely on devices which could get lost, stolen, or destroyed. Putting data in the cloud improves the user experience because recovery is simplified and users have more access to their data.
DataStore
App Engine Datastore is a schemaless object datastore providing robust, scalable storage for your web application. This means that you can’t run MongoDB, for example, on the Google App Engine (GAE). You need to use the Google datastore. However, GAE now supports Google Cloud SQL a fully managed MySQL service hosted on Google Cloud Platform.
The Go runtime environment
With the Google App Engine for Go, you can build web applications using the Go Programming Language. Your Go application runs on Google’s scalable infrastructure and uses large-scale persistent storage and services.
- App Engine builds and executes Go application code using a safe “sandboxed” environment. Your app receives web requests, performs work, and sends responses by interacting with this environment.
- The Go runtime environment uses Go version 1.4. The SDK includes the Go compiler and standard library, so it has no additional dependencies. As with the other runtimes, not all the standard library’s functionality is available inside the sandbox. For example, attempts to open a socket or write to a file will return a ‘os.ErrPermission’ error.
- The SDK includes an automated build service to compile your app, so you’ll never need to invoke the compiler yourself. And your app will be automatically rebuilt whenever you change the source.
- Go apps run inside a secure “sandbox” environment with a reduced set of libraries. For instance, an app cannot write data to the local file system or make arbitrary network connections. Instead, apps use scalable services provided by App Engine to store data and communicate over the Internet.
Note: I would love your feedback on these study notes. If I can make it better, I’d love it!
