Creating Google Cloud Pub/Sub publishers and subscribers with Spring Cloud GCP — Part 2: Implementation

Image from https://twitter.com/GCPcloud/status/962128994798841858

In Part 1 I talked a little bit about how we like to use Google Cloud Pub/Sub here at QuintoAndar and then showed how to setup a Spring Boot application powered by Spring Cloud GCP. Now we're going to see how to use it to implement Pub/Sub publishers and subscribers that help us develop cool message-driven micro-services.

But first, we need to understand the basics of how Cloud Pub/Sub works, so let's start talking about that.

Google Cloud Pub/Sub

In the previous article, we created a topic, which will have messages posted to it. Then, we added a subscription to that topic, which will have such messages delivered to. Pub/Sub makes sure that every subscription of a topic receives all the messages posted to it at least once.

But what about the publishers and subscribers? What do they do?

A publisher is responsible for posting messages to a topic. Here at QuintoAndar, for example, we have a service which handles the negotiations between landlords and prospective tenants. When such service needs to inform other services that an agreement was reached, it acts as a publisher and posts a success message to a specific topic. Every other service that is interested in knowing when there is an agreement, just need to create a new subscription to such topic.

A subscriber, on the other hand, is responsible for receiving and interpreting messages from a subscription. Pub/Sub provides us with two types of subscriptions: pull and push. In the pull subscription, the subscriber app calls a pull method which will be responded with a message and an acknowledge id. Then, the subscriber must call the acknowledge method to mark the message as received. In the push subscription, however, Pub/Sub is the one to start the delivery process, by calling an agreed endpoint on the subscriber; a success HTTP status code marks the message as received.

This article covers only the pull subscription, and we'll demonstrate one way of implementing a Pub/Sub publisher and a subscriber using Google Cloud GCP.


The PubSubTemplate class

The first thing we'll do is talk about the PubSubTemplate. It is, as stated on its Spring docs:

The main Google Cloud Pub/Sub integration component for publishing to topics and consuming messages from subscriptions asynchronously or by pulling.

So how does it help us?

It provides us with a really simple way of publishing and consuming Pub/Sub messages, by means of the methods:

PubSubTemplate publish and subscribe method signatures

The subscriber

As you can see in the signature of the subscribe method, in order to create a subscriber you need to have a subscription and a Consumer which expects a BasicAcknowledgeablePubsubMessage object as input. So the first thing we'll do is create an abstract class to represent a generic Pub/Sub consumer:

Now for every Pub/Sub subscriber we just need a concrete class that extends from PubSubConsumer and it should have all the needed parameters!

For example, to consume messages from the hello-pubsub-subscription we'll create a child class from PubSubConsumer called HelloPubSubConsumer. The consumer logic will be very simple, we'll just log the consumed message, but you can process it in any way you need.

Great! Now we've established how we want to consume messages from the hello-pubsub-subscription, but we are actually not doing anything yet. We still need to tell our app to use our consumer implementation to consume the messages it receives from the provided subscription. And that is where PubSubTemplate enters for the first time.

The strategy we're going to use is: we're going to create a configuration class and make it listen to a Spring Boot's ApplicationReadyEvent, which is published when the application is ready to receive requests. When an event of this type is captured, we know it's time to subscribe, so we're going to use PubSubTemplate to subscribe using our consumer implementation.

And just like that you have implemented your first Pub/Sub subscriber using Spring Cloud GCP! To test it, first you need to build and run your app:

# build
$ ./gradlew clean build
# run
$ ./gradlew bootRun

If everything is correct, you should see your subscriber subscribing to Pub/Sub right after your app starts running. It logs something like:

Now, with your app still running, you can use Google Cloud SDK (the same from part 1) once again to publish a message to your topic:

Which should log in your application something like:

message received: Hello, Pub/Sub

Now that you're ready to start consuming messages from a Pub/Sub topic, let's check out how to start producing Pub/Sub messages!


The publisher

Once again, we're gonna start checking the PubSubTemplate class for the signature of the publish method. As you can see, it expects a topic to publish to (which is just a String with its name), and a payload (which can be any Java Object) to be serialized and sent as a Pub/Sub message to the topic.

So we will start just like we did for the consumer, and create an abstract class to represent a generic Pub/Sub publisher. It should contain a topic to be defined by each of its concrete children and a publish method, which will use PubSubTemplate to publish the input message. Just to simplify, we'll assume the message will be a String, okay?

And then we'll create a concrete class extending from PubSubPublisher which will publish messages to the hello-pubsub topic.

Great! Now how can we test it? There are some alternatives, but a very simple way of doing that is exposing an endpoint which will trigger the publisher, so testing it becomes as simple as performing an HTTP POST request. Let's do that, shall we?

First things first, we'll start by adding Spring Boot Web Starter as a dependency for our project. It provides us with tools for creating web applications in a very simple way. To do that, go to your build.gradle file on your project's root and add the following to dependencies:

implementation('org.springframework.boot:spring-boot-starter-web')

That should be enough to have a local web application running in your local 8080 port when you run your project.

Now what we're going to do is create a mapping to the /hello/publish path to expect a POST request containing a message to be published in its body. The specifics of how its done is out of scope in this article, so I'll just put it here, but if you do want to understand more about creating REST services in Spring, I recommend this tutorial.

Pretty simple, right? Now build and run your project once again and you should be ready to test it!

To do that, we will make a POST request to the localhost:8080/hello/publish URL with the message in its body. I will use cURL for that, but you can use wget, Postman or whatever other tool you like to perform HTTP requests. To use cURL, just open a new terminal and run:

curl -X POST \
http://localhost:8080/hello/publish \
-H 'Content-Type: text/plain' \
-d 'Hello, Pub/Sub'

And that's it, there you have your first pair of Pub/Sub publisher and subscriber using Spring Cloud GCP! You can check out the full code here :D


Conclusion

In this second part, you learned:

  • What are the Cloud Pub/Sub publishers and subscribers and how they relate to its topics and subscriptions;
  • How the two types of Pub/Sub subscriptions (pull and push) work;
  • What is the PubSubTemplate from Spring Cloud GCP and how to use it to publish and subscribe;
  • To create working implementations of Pub/Sub publishers and subscribers;
  • How to attach subscribers as soon as the Spring Boot application is ready;
  • How to trigger Pub/Sub publishers using a REST API with Spring Boot Web Starter.