Task queues allow you to offload background work from your service by adding requests to a queue to be executed in the future. Using a managed service like Cloud Tasks has a lot of advantages, but it can be difficult to mimic the resulting task request coming from the queue when developing your worker services.
ngrok is a tool to expose a local web server to the public internet with a handy UI to inspect requests sent to the server and the server’s response. In this tutorial, we will learn how to use ngrok to locally test a pipeline of adding tasks to a Cloud Tasks queue and processing task requests with a worker service.
** This method has inherent risks associated with exposing your network externally. In this tutorial we will use Google Cloud Shell’s sandbox environment to minimize these risks. This process can be replicated on your local machine with the risks mitigated by not revealing your public URL and minimizing the time your tunnel is exposed.**
Before you begin
- Create or select a Google Cloud Platform project.
- Add a Google App Engine application to your project.
- Enable the Cloud Tasks API and Datastore API.
- Open Cloud Shell Editor. Cloud Shell provides built-in authorization for access to projects and resources hosted. If you are using a local machine, set up authentication to the API by creating a service account and setting the key as an environment variable.
- In the Cloud Shell terminal:
- Download and unzip ngrok:
wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zipunzip ngrok-stable-linux-amd64.zip
- Install the Python dependencies:
pip install --user flask google-cloud-datastore google-cloud-tasks
Create a queue
First, let’s make a queue to hold and send your task requests. Cloud Shell has pre-installed the Cloud SDK, which gives you access to the
gcloud command-line tool.
- Create a queue named “my-queue” using the following command:
gcloud tasks queues create my-queue
2. Make a note of some information about your queue: the project, the location, and the name. Identify these values by using the
gcloud tasks queues describe my-queue# Output
Start your server
Second, we will run a Python Flask app with endpoint
/update_counter that will receive Cloud Tasks requests and update a value stored in Cloud Datastore.
- Create a file named
main.pyand copy over the following code:
2. In the Cloud Shell terminal, start the app server using the following command:
The Flask app is set to run on port 8080. We will also set ngrok to listen to this port.
We will need to start ngrok before we create our first task.
- Open another tab in your Cloud Shell terminal (➕) and start ngrok with the following command:
./ngrok http 8080
2. Make a note of the forwarding URL. In this example, the URL is https://2da88316.ngrok.io.
3. ngrok provides a real-time web UI, on port 4040, where you can inspect all of the HTTP traffic running over your tunnel. To open the inspection UI, click the Web Preview button in the top right corner, then select “Change port”. Enter 4040 and click “Change and Preview”. This opens http://localhost:4040 in your browser from Cloud Shell.
Create a Task
Now, let’s create and add a task to your queue.
- Create a file named
create_http_task.pyand copy the following code:
2. Update variables project, queue, location, and url with the values noted above:
project = ‘my-project-id’
queue = ‘my-queue’
location = ‘us-central1’
url = ‘https://2da88316.ngrok.io/update_counter'
Make sure your URL retains the “https” and the endpoint “/update_counter”, so the request can be properly routed to the handler.
3. Now send a create task request targeted at the ngrok public URL:
4. Look at the inspection UI. If the set up was correct, you will see a 200 OK response
You can also look at the details of the request and response, including the time, duration, headers, query parameters and request payload as well as the raw bytes on the wire.
You can replay the request in the inspection UI in order to reduce API calls.
Customize the request body and headers
The Cloud Tasks API enforces the task request body to be encoded as a byte array. The request body is added to the task dictionary under
create_http_task.py, the string variable
payload has been encoded to a byte array with the method
Try changing the Content-Type of the request, from the default “application/octet-stream”, by adding the headers field to the
http_request dictionary. The Flask route can parse the request body based on this Content-Type.
Using Google App Engine targets
You must use HTTP targets in order to make use of ngrok’s public URL. Though you can easily change
relative_uri to target an App Engine endpoint.
- Shutting down your ngrok instance using
CTRL+Cin the terminal.
- Shut down your local Python server using
CTRL+Cin the terminal.
- To avoid incurring charges to your Google Cloud Platform account for the resources used, go to the console and delete your project.
Learn more about the Cloud Tasks service in the Cloud Tasks Overview.