Google Cloud Pub/Sub Reliability Guide: Part 3 Administrative Operations

This post is part of a series to will help users of Google Cloud Pub/Sub write reliable applications that use the service. I recommend reading through Part 1 before starting on this one.

Yes, Pub/Sub is very reliable and highly available. No, it is not perfectly reliable. And that is where you, as an application developer, come in. My hope is that this set of articles will give you the background to design for extreme reliability. The articles are written by the product manager of Cloud Pub/Sub with a lot of help from Kamal Aboul-Hosn and others.

Administrative operations

Administrative operations, such as creation of topics and subscriptions, are different from data operations in several respects:

• They are strongly consistent: Unlike message publishing or subscribing, creation and deletion of resource is strongly consistent relative to the list and get operations. It is also strongly consistent in that attempting to publish or subscribe to a newly created resource will always work predictably. At the same time, resource metadata can be cached order of minutes. This means, for example, that if you delete and recreate a subscription with the same name, you may get messages meant for the older subscription for some time.
• They take longer: these global operations may take tens of seconds to complete under extreme conditions.
• They fail more frequently: while data access operations are designed to be extremely reliable, administrative operations are presumed to be comparatively rare. Therefore you should expect these operations to be highly reliable, but not as reliable as publishing and subscribing.
• They are independent of the data operations: when you are unable to create or list resources in a given region, you will generally be able to subscribe and publish to these resources. The reverse is also true: if the service is suffering publish or subscribe unavailability in a given region, you are likely to be able to issue administrative operations successfully.

While Pub/Sub administrative operations have generous quotas of hundreds of resource creations per second per project, designs that require frequent resource creation and destruction should be considered with care. In addition to the eventual consistency effects mentioned above, there are hard limits on the total number of topics and subscriptions per project. Reliance on frequent creation of resources may cause your application to hit these limits.

Next Steps

This concludes the initial installment of the series. You may wish to review Parts 1 and 2 covering Publishing and Subscribing. And as always, there is much to be made of the product documentation. Your comments and feedback are read and appreciated.