Migrating from GKE to Cloud Run

Migrating a golang REST API from Kubernetes Engine (GKE) to Cloud Run

Migrating to Cloud Run

Background

Kubernetes at Google Cloud (GKE) was used during the development and the testing of the Dreamwod app (https://www.dreamwod.app). GKE has been working good but it has also been the most expensive and complicated part of the hosting. We have because of that been migrated from GKE to Cloud Run. The primary reasons behind the migrations are.

1. Cost - Cloud Run is cheaper than GKE (for our workload).
2. Simplicity - Our API is pretty simple and we don’t need a lot of the extra features in Kubernetes.

Migrating secrets

When migrating to Cloud Run we decided to store our secrets in Google Cloud Secret Manager. There isn’t a native Cloud Run integration with Secret Manager so the recommended way forward is to load any secrets when the container is starting. We decided to store the whole .yml config file in one secret so we can start the container after only one request to Secret Manager.

Loading configuration from Google Cloud Secrets Manager

Migrating Cloud SQL

We used cloud-sql-proxy as a sidecar when connecting to Cloud SQL from Kubernetes. The cloud-sql-proxy functionality is already built-in into Cloud Run so the only thing that needs to be done is to add the connection and to update the connection string. On Kubernetes we used the following configuration and the connection string postgresql://username:password@127.0.0.1?sslmode=disable.

Cloud SQL Proxy example

On Cloud Run we used the following connection string postgresql://username:password@:5432/database-name?host=/cloudsql/<instance-connection-name>

Migrating asynchronous tasks

We had certain functionality when we for example did processing of images in the background using goroutines. That had to be changed since Cloud Run throttle the CPU if no requests are active.

The new approach is instead to send messages to Cloud Pub/Sub when we need to do background processing and then have a subscription that pushes messages back to the API. The below code publishes events of different types to Pub/Sub.

Code that publishes events to Pub/Sub

Migrating the build pipeline

We use Cloud Build for building and deploying containers. Deploying with Cloud Run is just adding one extra step to the build pipeline.

Migrating DNS

We added a subdomain mapping in Cloud Run, updated the DNS and the migration was finished.

Updating logging

We use gin-gonic as the web framework and Zap for logging.

1. Request logging was disabled in gin-gonic since Cloud Run is logging all requests.
2. Zap is updated with below to make Cloud Logging pick up the severity.

Conclusion

Performance

We don’t have a lot of load on the API but we haven’t been able to see any performance degradations after the move to Cloud Run.

Startup time

The startup time for processing a request when the instance is cold is impressive. For our API it’s between 1 and 2s. That includes starting the container, doing a database migration/update check and loading configuration from Secret Manager.

Cost

A typical month for us (everything hosted in Amsterdam datacenter) with Kubernetes was around $70/month.

1. Compute Engine $58 (84%)
2. Cloud SQL $11 (15%)
3. Others $1 (1%)

When running on Cloud Run we are currently in the free tier which means that the cost for Cloud Run is zero. The total cost is around $12/month.

1. Cloud SQL $11 (91%)
2. Others $2 (9%)