Announcing Relé!

Andrew Graham-Yooll
Nov 5, 2019 · 3 min read
Relé — A powerful Python integration for Google Cloud’s Pub/Sub

We are pleased to announce Relé, a comprehensive Python solution to get you up and running with Google Cloud’s Pub/Sub as quickly and easily as possible. Meant to ease development workflow, Relé comes built-in with key components for managing a project with Pub/Sub enabled services.

Out-of-the-box, Relé provides:

  • Powerful Publishing API
  • Highly Scalable Worker
  • Intuitive Subscription Management
  • Easily Extensible Middleware
  • Optional Django Integration
  • And much more!

Allowing our products to scale with ease, Pub/Sub is a vital piece of infrastructure at Mercadona Tech and we lean on Relé to manage publishing and subscribing to millions of messages.

Use Case

For many years, Python’s go-to task queuing library has either been Celery or RQ. Although on the face of it, Pub/Sub’s design may look similar, it is solving a different problem.

While Celery and RQ solve task queues within an application, Pub/Sub shines when it comes to managing task queues between many services. Leveraging Relé with Pub/Sub maintains an elegant API much like Celery and RQ’s, but instead allows for scalable messaging within a service oriented architecture.

Reasoning

Since the Google Pub/Sub client does not come built-in with solutions to database connection management, publisher management, declaring subscribers, and running workers, the Mercadona Tech team decided to coalesce around an open source solution.

To reduce complexity and risk when adopting Pub/Sub, we organized our experiences from the Python client into one comprehensive library to solve common use cases and patterns suitable for a highly-available production environment.


Getting Started

pip install rele

Or with Django:

pip install rele[django]

Configuring

For Relé to work, we must have a Cloud Pub/Sub topic. Via the Google Cloud Console we create one, named photo-uploaded.

Relé configuration looks like:

Once the topic is created and our application has the proper configuration defined, we can start publishing to that topic.

Publishing

To publish, we simply pass in the topic to which we want to publish too, followed by a valid JSON serialize-able Python object.

Subscribing

Once we can publish to a topic, we can subscribe to the topic from a Relé worker.

You will notice the simple API for both publishing and subscribing. We have taken inspiration from numerous other libraries such as Celery.

Consuming

Once the sub is implemented, we can initialize our worker. This will register the subscriber with Google Cloud and will handle pulling messages from the topic python worker.py.

If you are integrating Relé into your Django project, declaring workers are not necessary. Since the provided python manage.py runrele command will auto-discover your subs. The only thing necessary is modifying your settings.py file.

The code for these example can be found here.


We hope that Relé provides you and your project the ability to leverage Google Cloud’s Pub/Sub with ease.

For more information and guides, please see our docs or submit an issue.

And do not forget that contributions are always welcome!

Mercadona Tech

Somos el equipo detrás de la nueva experiencia online de…

Thanks to Miquel Torres

Andrew Graham-Yooll

Written by

Mercadona Tech

Somos el equipo detrás de la nueva experiencia online de Mercadona.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade