Backend in Dart: Creating a Pub/Sub handler in GKE

11 min readFeb 6, 2024

Backend in Dart brings a lot of advantages to Flutter developers. You can reuse code between the backend and frontend. And it’s just easier to switch your attention between the two.

There are many tutorials on using shelf server to just respond to HTTP requests. Much less is written on microservices.

One of the patterns there is “Publish-subscribe”. It allows one piece of your backend to publish a message to later be processed by another piece, a dedicated microservice. This way, the original synchronous request can be processed faster by shifting part of the work into an asynchronous processor. This shifted work can be logged, retried, throttled, and load-balanced without the original processor knowing.

This is heavily used to aggregate statistics, send notifications, clean up, for heavy computation, and anything that may take a long time or need retries.

A handy implementation of the “Publish-subscribe” pattern is “Pub/Sub” service in Google Cloud.

Example

In this example, we will create a simple Pub/Sub handler that receives a message, capitalizes its value, and sends it to another Pub/Sub topic:

To make the experience complete, we will progressively set up this architecture:

We will walk through each of these layers.

Setting

1. Clone the project

I have this project checked out as

/home/user/dart-pubsub-gke-demo

Replace it with your path in all commands.

2. Create a Google Cloud project

Create a project here: https://console.cloud.google.com/projectcreate

3. Create Pub/Sub topics and subscriptions

Go to Pub/Sub: https://console.cloud.google.com/cloudpubsub/topic/list

Create two topics named input and output, with default subscriptions. Set one-day retention.

4. Create a service account

Create a service account named capitalizer here:
https://console.cloud.google.com/iam-admin/serviceaccounts/create

Click “Create and Continue”:

Grant three roles:

Pub/Sub Subscriber
Pub/Sub Publisher
Pub/Sub Viewer (see and upvote this issue)

Finish the account creation. Open the account, create and download its JSON key:

Put the key under this name:

/home/user/dart-pubsub-gke-demo/keys/capitalizer.json

This is it for the setup.

Step 1. Create the Pub/Sub handler in Dart

The handler

The code of the handler is in main.dart:

const _subscriptionName = 'input-sub';
const _topicName = 'output';

Future<void> main() async {
  final projectId = Platform.environment['PROJECT'];
  print('Project ID: $projectId');

  if (projectId == null) {
    throw Exception('Set PROJECT environment variable.');
  }

  final pubsub = PubSub(
    await clientViaApplicationDefaultCredentials(scopes: PubSub.SCOPES),
    projectId,
  );

  final inputSubscription = await pubsub.lookupSubscription(_subscriptionName);
  final outputTopic = await pubsub.lookupTopic(_topicName);
  print('Looked up: $inputSubscription, $outputTopic');

  while (true) {
    print('Pulling.');

    final event = await inputSubscription.pull();
    print('Event: $event');

    if (event == null) {
      print('Idle.');
    } else {
      final text = event.message.asString;
      print('Message received: $text');

      final data = jsonDecode(text) as Map<String, dynamic>;
      final value = data['value']?.toString() ?? '';

      final response = {...data, 'value': value.toUpperCase()};

      await outputTopic.publish(Message.withString(jsonEncode(response)));
      await event.acknowledge();
    }

    await Future.delayed(const Duration(seconds: 5));
  }
}

In a terminal session, set PROJECT environment variable to the ID of your project for all future commands:

export PROJECT=your-project-id

To try the handler locally, run:

cd /home/user/dart-pubsub-gke-demo/capitalizer

GOOGLE_APPLICATION_CREDENTIALS='../keys/capitalizer.json' dart lib/main.dart

Here, GOOGLE_APPLICATION_CREDENTIALS is the environment variable used by Google Cloud client libraries to authenticate. Because of that, you don’t handle the key explicitly in your Dart code. This is explained here.

Manual test

You can now publish a message to input topic and see it printed in the terminal:

The terminal will show you the message:

$ GOOGLE_APPLICATION_CREDENTIALS='../keys/capitalizer.json' dart lib/main.dart 
Project ID: your-project-id
Looked up: Instance of '_SubscriptionImpl', Instance of '_TopicImpl'
Pulling.
Event: null
Idle.
Pulling.
Event: null
Idle.
Pulling.
Event: Instance of '_PullEventImpl'
Message received: {"value": "Hello!"}
Pulling.
Event: null
Idle.
Pulling.

To see the output in the console, open output topic, go to messages, select output-sub subscription, and click “Pull”:

Automated test

The test is in main_test.dart.

It publishes a message into input topic and expects the output from output-sub subscription.

Run it like this:

$ GOOGLE_APPLICATION_CREDENTIALS='../keys/capitalizer.json' dart test 
00:00 +0: test/main_test.dart: Publish and read the result                                                                                                            
Project ID: your-project-id
Purging the output subscription.
Event: Instance of '_PullEventImpl'
00:11 +1: All tests passed!

Stopping the service

In the terminal, press Ctrl-C to terminate the process.

Step 2. Create a Docker container

Most of the production scenarios include packing this application into a Docker container.

Build the container for the local run

For this, the repository includes Dockerfile. Run:

docker build \
  -t capitalizer \
  --build-arg DART_VERSION=3.2.0 \
  .

Notice how small the image is:

$ docker image list | grep capitalizer
capitalizer               latest    68af730f34e1   16 seconds ago   10MB

Run the container locally

To try the container locally, run:

docker run \
  -it \
  --rm \
  -e "GOOGLE_APPLICATION_CREDENTIALS=/app/keys/key.json" \
  -e "PROJECT=$PROJECT" \
  -v /home/user/dart-pubsub-gke-demo/keys/capitalizer.json:/app/keys/key.json \
  --name capitalizer \
  capitalizer

Testing

Since the test communicates to “Pub/Sub” service in Google Cloud, it works regardless of how the app is run. When the container is running, you can test it just as before, both through the Google Cloud console and dart test.

Stopping the container

For some reason, I could not terminate the process with Ctrl-C on Mac. Run this in another terminal session to stop the container:

docker stop capitalizer

Step 3. Run in Kubernetes locally

One of the most convenient ways to deploy and run Docker containers is Kubernetes. It allows you to start and stop containers and to keep your key securely.

Before deploying your container into a cloud, you can try it on a local Kubernetes instance to see how it behaves.

1. Install Minikube

Follow the instructions here: https://minikube.sigs.k8s.io/docs/start/

2. Start Minikube

minikube start

3. Create a secret

kubectl create secret generic capitalizer \
  --from-file=key.json=/home/user/dart-pubsub-gke-demo/keys/capitalizer.json

Here, capitalizer is the name of the secret. A secret in Kubernetes may contain multiple name-value pairs. Here we only use one. The name is key.json, and the value is the content of the file. When a secret is mounted to a container, it creates a directory, and each name-value pair becomes a file name and its content. This is why the name key.json is here, earlier in Dockerfile we hardcoded the key under that name.

4. Inspect the secret

kubectl describe secret capitalizer

It should show you something like this:

Name:         capitalizer
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
key.json:  2357 bytes

5. Re-build the container using Minikube’s Docker

Minikube uses its own Docker instance that runs in its virtual machine. It can’t see the container you created earlier.

First, make Minikube show you how to access its Docker. Run:

minikube docker-env

It will print something like this:

export DOCKER_HOST="tcp://127.0.0.1:12345"
export DOCKER_CERT_PATH="/home/user/.minikube/certs"
export MINIKUBE_ACTIVE_DOCKERD="minikube"

Copy it and run it in your local terminal to create those environment variables. From that point on, the docker command will talk to the instance of Docker that Minikube has created. To revert that later and talk to your primary Docker, start a new terminal session.

Finally, build the container so that Minikube sees it:

docker build \
  -t capitalizer \
  --build-arg DART_VERSION=3.2.0 \
  .

6. Apply the deployment

This should automatically start the container:

envsubst < deployment_minikube.yaml | kubectl apply -f -

Note that the project ID is required for the container, and deployment_minikube.yaml has a substitution for it from the environment variable. Kubernetes can’t handle that substitution, so we use envsubst on this file and then feed the final YAML with the project ID to kubectl apply.

7. Check the status of the pod

kubectl get pods

It should show you something like this:

NAME                           READY   STATUS    RESTARTS   AGE
capitalizer-79f6bf9fcc-5w9rb   1/1     Running   0          7s

8. Inspect the logs

The status of the pod does not guarantee your app is running as expected within it. To view the stdout of the app:

kubectl logs $(kubectl get pods -o name | grep dart-pubsub-gke-demo | head -n 1)

It should show you the output of the Dart code:

Project ID: your-project-id
Looked up: Instance of '_SubscriptionImpl', Instance of '_TopicImpl'
Pulling.
Event: null
Idle.
Pulling.
Event: null
Idle.
Pulling.

9. Shutdown the pod

One of the ways is to delete the deployment:

kubectl delete deployment dart-pubsub-gke-demo

Note that we use dart-pubsub-gke-demo for the name of the deployment and capitalizer for the name of the microservice and container. This is because in a real project you will have multiple microservices. To avoid confusion, don’t name them the same as the entire system.

Step 4. Run in Google Kubernetes Engine

Time to get the app to the cloud.

1. Install gcloud CLI

You will use it to run commands.

1. https://cloud.google.com/sdk/docs/install

2. https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke

2. Enable Artifact Registry

To start a container in the cloud, its image also must be in the cloud. There are a lot of options to store images, and a convenient one is Google Artifact Registry. Enable it for your project here.

3. Create an artifact repository

Create a repository with “Docker” as the format. Name it my-repository, choose any region, and leave everything else at the default values.

4. Create environment variables for region and repository

They will be used a lot, just like PROJECT was used earlier. Create the variables using the values you entered earlier:

export REGION=us-central1
export REPOSITORY=my-repository

5. Tag the image for Artifact Registry

Full help article:
https://cloud.google.com/artifact-registry/docs/docker/pushing-and-pulling

Re-build the image with the tag for Artifact Registry:

docker build \
  -t $REGION-docker.pkg.dev/$PROJECT/$REPOSITORY/capitalizer:v1.0.0 \
  --build-arg DART_VERSION=3.2.0 \
  .

6. Create a service account for deploy

Create an account named deploy. Give it these roles:

Artifact Registry Writer
Kubernetes Engine Developer

Create a key for the service account and put it under

/home/user/dart-pubsub-gke-demo/keys/deploy.json

7. Push the image to Artifact Registry

gcloud auth activate-service-account --key-file=../keys/deploy.json
gcloud auth configure-docker $REGION-docker.pkg.dev --quiet
docker push $REGION-docker.pkg.dev/$PROJECT/$REPOSITORY/capitalizer:v1.0.0

8. Verify the image in Artifact Registry

Check if the image has appeared in the Google Cloud console, or run:

gcloud artifacts \
  docker images list $REGION-docker.pkg.dev/$PROJECT/$REPOSITORY

9. Enable Google Kubernetes Engine

Here: https://console.cloud.google.com/marketplace/product/google/container.googleapis.com

10. Create a new cluster with the default settings

Create a standard cluster (not autopilot).

Name it my-cluster. The default zone us-central1-a will do. Set 1 node to make your experiment cheaper, and leave everything else at the defaults:

11. Create environment variables for cluster and zone

A zone is a more narrow thing than a region that the artifact registry uses, so it needs a separate variable:

export CLUSTER=my-cluster
export ZONE=us-central1-c

12. Make kubectl use the cluster

gcloud container \
  clusters get-credentials $CLUSTER --zone=$ZONE --project=$PROJECT

From now on, kubectl command will be working with the GKE cluster. To make it point to your local Minikube again later, stop and start Minikube.

13. Create a secret in GKE

kubectl create secret generic capitalizer \
  --from-file=key.json=/home/user/dart-pubsub-gke-demo/keys/capitalizer.json

Inspect it as you did with the local secret:

kubectl describe secret capitalizer

14. Apply the deployment

VERSION=v1.0.0 envsubst < deployment_gke.yaml | kubectl apply -f -

Note that deployment_gke.yaml is used, a different configuration from your local one. It uses a fully qualified image name to pull the container from Artifact Registry.

15. Check the status of the pod

kubectl get pods

Ideally you should see the same as with the local test earlier:

NAME                           READY   STATUS    RESTARTS   AGE
capitalizer-799b786987-4zhzk   1/1     Running   0          7s

And then inspect the logs to verify the stdout of the Dart code.

But if your local platform is different from the GKE cluster node, you will have an error like this:

NAME                          READY   STATUS             RESTARTS         AGE
capitalizer-59f75c59f-sdz62   0/1     CrashLoopBackOff   11 (3m48s ago)   35m

This happened to me on Mac. Verify the cause of the error:

kubectl logs $(kubectl get pods -o name | grep dart-pubsub-gke-demo | head -n 1)

In case of a platform mismatch, the output will be:

exec /app/bin/server: exec format error

In this case, you should use Google Cloud Build to build an image for the target platform (see below).

16. Clean up the deployment

kubectl delete deployment dart-pubsub-gke-demo

Note that this does not destroy the GKE cluster which will still consume your money. Delete the cluster in the console if you are taking a long break. The screenshot earlier showed it consumes about $0.15/hour.

Step 5. Use Google Cloud Build to build images

You should use Cloud Build if the images you build locally do fail to run on your GKE nodes.

But even if they do run, Cloud Build offers better security. Allowing locally built images into Artifact Registry is bad for production security because images are opaque when they are pushed. Your local malware can get into the cloud unnoticed. If you only allow images from Cloud Build, you reduce the attack surface.

1. Enable Cloud Build

Here: https://console.cloud.google.com/marketplace/product/google/cloudbuild.googleapis.com

2. Add roles to ‘deploy’ service account

Add:

Cloud Build Editor
Storage Admin
Viewer (if you want to see the building progress in terminal)

3. Build

gcloud builds \
  submit \
  --project=$PROJECT \
  --substitutions="_VERSION=v1.0.1,_DART_VERSION=3.2.0,_REGION=$REGION,_REPOSITORY=$REPOSITORY" \
  --config=cloudbuild.yaml \
  .

Note the bumped version from the previous section.

Wait until the command completes.

4. Apply the deployment

VERSION=v1.0.1 envsubst < deployment_gke.yaml | kubectl apply -f -

5. Check the status of the pod

kubectl get pods

This time it should be running no matter your local architecture.

NAME                          READY   STATUS    RESTARTS   AGE
capitalizer-854999988-7mhtl   1/1     Running   0          10s

6. Inspect the logs

To view the stdout of the app:

kubectl logs $(kubectl get pods -o name | grep dart-pubsub-gke-demo | head -n 1)