Creating Google Cloud Pub/Sub publishers and subscribers with Spring Cloud GCP — Part 1: Setup
Google Cloud Platform offers us a lot of great tools for cloud computing, and here at QuintoAndar we have been exploring them for the past year. One of the tools we like a lot is Cloud Pub/Sub, which provides us with resilient and scalable message-oriented communication among our microservices.
When we first started using Pub/Sub, we created our own implementations of its publishers and subscribers using Google's Java client libraries. However, as of this year, Google Cloud and Spring announced Spring Cloud GCP, which makes it a lot easier to integrate our Spring-based Java back-end servers with GCP.
In this article, I'll focus on how to use this powerful tool to develop fast and simple Cloud Pub/Sub publishers and subscribers using Spring Cloud GCP. The whole code can be found at https://github.com/fmachado091/spring-cloud-gcp-pubsub-example.
The Google Cloud Platform project
We'll use the Google Cloud SDK to setup our project in GCP. Follow these instructions to install the latest Google Cloud SDK in your environment (but no need to run
gcloud init at the end, we'll do it soon).
For this tutorial, we'll create a new GCP project (I called it
hello-spring-cloud-gcp, but this name needs to be unique, so choose a new one for yours). This project will publish and subscribe to a Pub/Sub topic called
hello-pubsub through a Pub/Sub subscription called
P.s. If you're more of a graphical user interface type of person, all of the GCP setup can be done at GCP's console.
The Spring Boot project
Now we’ll create a new Spring Boot Gradle project, built on top of a Java environment. Spring Initializr provides us with a very simple way of bootstrapping our application. We just need to choose our tools and add the dependencies we need (in this case, only GCP Messaging).
Before going on, you may want to edit your
build.gradle file a bit, just for setting the Java and Gradle versions for your project. In this article, we'll be using Java 11 and Gradle 4.10.2, but every Java version starting on Java 8 should be fine. Also, the project comes with a Gradle wrapper in it, so you don't need to locally install it if you don't want to. You can check out the final version here.
Now, we're going to try to build our app for the first time. In the project's root directory, run
./gradlew clean build, and it should prompt an error just like this:
br.com.quintoandar.springcloudgcppubsubexample.SpringCloudGcpPubSubExampleApplicationTests > contextLoads FAILED
Caused by: org.springframework.beans.factory.BeanCreationException
Caused by: org.springframework.beans.BeanInstantiationException
Caused by: java.lang.RuntimeException
Caused by: java.io.IOException
And if you check the log of this failed test, you should see something like this:
Caused by: java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
It happens when it fails to authenticate the app with valid GCP credentials, which is expected at this moment since we haven't added our GCP project's credentials to our app yet. But fear not, dear reader, for we shall do that in the next step!
Add GCP credentials to Spring Boot
To do that, we need to create a new service account for our project. A service account is a type of Google account that belongs to an application instead of an user, so we need one to authenticate our app. Once again, we're going to use Google Cloud SDK to help us with that.
After you follow these steps, you should have downloaded a
.json file with your project's credentials to your local machine.
Now there are two basic ways of authenticating your Spring Cloud GCP app with a service account key file. The first one is to add the full path to the key file to the GOOGLE_APPLICATION_CREDENTIALS environment variable:
$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/keyfile.json
$ ./gradlew clean build
The second (and my personal favorite, since you don't need to store the actual file anywhere) is to make the appropriate Spring properties contain you key file info. To do that, you need to encode your key file into Base64 format. Now you can either use an app such as Base64Encode.org to help you with that or use your own OS's tools. For instance, in Linux systems:
$ cat /path/to/your/keyfile.json | base64 -w0
Now whatever tool you use, just make sure you copy the result and add to your project's
application.properties file as:
You can also store your encoded key file in an environment variable. Just need to change your
application.properties to get the credentials property from it:
And that's it! Just run
./gradlew clean build once again in your root directory and you should see a beautiful
BUILD SUCCESSFUL message.
In this first part, you learned to:
- Setup Google Cloud SDK and use it to create a new GCP project with Pub/Sub enabled, to create Pub/Sub topics and subscriptions and to create a service account that identifies your app;
- Create a new Spring Boot project enhanced with Spring Cloud GCP;
- Authenticate your Spring Boot project with your GCP service account key file.
Now go ahead to Part 2 of this article and check out how to implement Pub/Sub publishers and subscribers using your new Spring Boot app.