Asynchronous Code Execution with Google Cloud Tasks
No application is an island — every non-trivial application has dependencies and exists alongside other services, sharing resources and data. Over the years, messaging has evolved from static, synchronous point-to-point models to asynchronous mechanisms.
In this blogpost, we’ll learn what Google Cloud Tasks, an asynchronous point-to-point queuing system for code execution, where you could benefit from using Cloud Tasks, and how to get started.
What is Cloud Tasks?
Cloud Tasks is a queuing system service for Google Cloud Platform for managing the execution, dispatch, and delivery of a large number of distributed tasks.
With Cloud Tasks, you can add millions of “Tasks” in a queue. You may also create many queues. The queue processes Tasks in the queue, sending an HTTP request to the target URL or executing an App Engine application handler, depending on your setup.
An example diagram of this relationship is below.
Right: A server listening to HTTP requests (our HTTP target)
Using Cloud Tasks can increase the reliability, manageability, and responsiveness of your applications. For example, Cloud Tasks works particularly well with decoupled services built with Cloud Functions and Cloud Run.
Cloud Tasks includes these features:
Flexible Routing
Dispatch tasks that reach any target within GCP and on-prem systems.
Tasks Offloading
Offload heavyweight, background and long running processes to a task queue, allowing an application to be more responsive to users.
Rate Limiting
Control the rate at which tasks are dispatched to your service to ensure your microservices doesn’t get overwhelmed.
High Reliability
With each task is persisted in storage and retried automatically, your infrastructure is resilient to intermittent failures.
Loose Coupling
Microservices often don’t talk to each other via direct request/response, but rather asynchronously, allowing services to scale independently. Cloud Tasks helps you better structure and scale your application via dedicated independent, configurable Task queues.
This allows you to do things like pause a Tasks queue, or configure independent retry policies.
Other features include scheduling a Task for up to 30 days, Queue rate limiting, configurable retries, Task deduplication, and Task handler auth.
Where could you use Cloud Tasks?
Cloud Tasks can be used to call Google App Engine, Cloud Run, Cloud Functions, or any HTTP server.
Here are some example use-cases:
- Push an asynchronous image processing job that retry.
- Read from a database at a controlled and consistent rate.
- Schedule an email to be sent 1 month after sign-up of a trial period.
So why would you use Cloud Tasks versus Pub/Sub?
Pub/Sub vs Cloud Tasks
The core difference between Cloud Pub/Sub and Cloud Tasks is the notion of implicit vs explicit invocation.
Cloud Pub/Sub supports implicit invocation: a publisher implicitly causes the subscribers to execute by publishing an event.
By contrast, Cloud Tasks is aimed at explicit invocation where the publisher retains full control of execution.
A detailed comparison and handy side-by-side comparison can be found here.
Tools of Cloud Tasks
How can you use Cloud Tasks? To control Cloud Tasks, you have a variety of tools, for manual and programmatic use:
- Google Cloud Console
gcloudCLI- Cloud Tasks API
Google Cloud Console
At console.cloud.google.com/cloudtasks, you are presented an interface for viewing all queues and controls for resuming/pausing the queue, deleting the whole queue or individual Tasks, as well as purging/deleting all Tasks.
The gcloud CLI
One of the easiest ways to use Cloud Tasks is by using the command line.
With gcloud, you can easily create Tasks with a command like such:
gcloud tasks queues create "example-queue" Ignore the warning about
queue.yamlandqueue.xml.
This creates a new queue which can contain thousands of Tasks.
Cloud Tasks API
Programmatically, we can use Tasks with Google API clients, for example, the Node client:
Thanks for reading! That’s a brief introduction to Cloud Tasks.
