Securing Cloud PubSub Push with Cloud Endpoints

Google Cloud PubSub is a fully managed service for real-time messaging and streaming data. Till now it was one of the most reliable components in our big data pipeline. But it’s less obvious that you can use it as a way to propagate changes throughout the system in a micro-service architecture.

While big data pipelines are in general high throughput, change propagation could be less chatty. For these scenarios PubSub has a nice feature called Push Subscriptions. Instead of writing specialised code in the pull model to pull messages from a subscription, a push subscription just let’s PubSub do a post on an HTTP endpoint.

PubSub Push

The PubSub Push Subscription feature takes a lot of the complexity out of the equation. No need to think about acknowledgments, just make sure you can handle the messages fast enough and return a 204. PubSub will make sure to keep the messages coming or back-off when your service can’t keep up. One caveat is that only one message is pushed at a time, but that’s ok for low throughput scenario’s. Remember: you don’t have to write any special code.

Create a new API key in the Cloud console

But, there is always a but… what about security? The PubSub FAQ hints on only accept messages that are accompanied by a secret key. But this implies writing extra code. Secret keys also means managing and rotating your keys, again extra code. Unless…

What if you could again move the key handling out of your code? Well you can, by using Cloud Endpoints.

Endpoints

Cloud Endpoints is a proxy you can put in front of your micro-service that can handle your authentication, monitoring, logging and tracing. It also has supports for API keys and is easy to use. The only thing you need to write for your endpoint is a OpenAPI file ( previously known as swagger ). The file needs to get pushed to your project using

gcloud service-management deploy openapi.yaml

In the example below we have a /api/hellopy/subscriptions/foobar push endpoint we want to secure. Simply by adding a security definition with a definition of api_key the endpoint proxy will handle all the key verification.

swagger: "2.0"
...
paths:
...
  "/api/hellopy/subscriptions/foobar":
    post:
      description: "PubSub push."
      operationId: "pubsub_foobar"
      produces:
      - "application/json"
      responses:
        204:
          description: "Ack"
      security:
        - api_key: []
...
securityDefinitions:
  api_key:
    type: "apiKey"
    name: "key"
    in: "query"

In the Cloud project you need to create a credentials API key in the API manager. The key can then be used as a query parameter in the subscription url. That’s it.

Use the API key in the URL

Now you can start testing your endpoints by pushing some messages on the topic. Make sure to keep and eye on the logs to see if your messages come through.

Aside the zero code solution, you also have the ability to rotate the key by pressing Regenerate key in the API manager. The API manager gives you a grace period of 24 hours in which you have time to update the keys in your subscriptions, it will keep accepting the original key during that time.

Conclusion

This is now my preferred way to secure my PubSub Push endpoints. I certainly like the no code approach, because you should resist writing your own security handling code. Making it full circle you could automate this process, including the key rotation. But we’re missing some API’s here. Updating the subscriptions via an API is easy, but we’re missing an API here for creating and rotating keys (Google are you listening?).

Apart from the missing API for maintaining you keys, you have now a good way to secure your PubSub push subscriptions without writing a single line of code.

Wish list for Google:

• Add an API for managing API keys
• Publish a well known list of IP ranges that PubSub could publish from

References

• PubSub subscriber guide: Read how to configure your subscription in Google Cloud PubSub
• PubSub FAQ on security: The recommendation on using tokes in the push url
• Cloud Endpoints for container engine: Quickstarter on using Endpoints on Kubernetes
• Credential support: How to create an API key in the credentials screen