Deploy ML model on Google App Engine by triggering Google Cloud Functions

8 min readNov 5, 2021

This article focuses on deploying ML model using Google cloud functions. If you are interested in deploying your Machine Learning model using Docker and Google App Engine without triggering Google cloud function, please visit here.

If you are interested in jumping straight to the code, please visit here.

Google Cloud Functions

Google cloud functions are popularly known as Functions as a Service(Faas). They are serverless service from GCP which doesn’t require you to manage servers and setup infrastructure. It scales up automatically and handles millions of invocations per day. In this article, we will have our model inference code deployed as a Google cloud function and have it triggered in response to image request from our Streamlit web app via REST API.

Please check out my article here to get a gist of what Google App Engine, Docker, Streamlit does.

Here are some pre-requisites that you need to consider before you begin :

Assuming you already have a GCP account. If you don’t have one, please sign up here which comes with $300 credit. Once you sign up, make sure to create a project.
Install google cloud SDK to interact with google cloud from your local.
Install Docker from here.
Install streamlit from here.
Trained ML model.

For brevity, I have generated a ML model which takes an image as input and outputs caption for that image. Similar code to train an image captioning model can be found here.

Step 1 : Upload model weights to Google cloud Storage

Next step is to Create a Bucket on Google cloud Storage.

Search for Cloud Storage under the Navigation menu. Choose a bucket name that is globally unique and make sure to choose the location where you want your cloud function to be run. You could leave the other settings to default.

Upload the saved model weights to the bucket. You could also upload them using command line.

Step 2 : Create Cloud Function

Look for Cloud Functions under the Navigation Menu and hit ‘Create Function’. Make sure to choose the region to be same as the bucket where your model weights reside. For the purpose of this project, we have set ‘Allow unauthenticated invocations’ under Trigger section.

Since we use Transformers to generate captions, we will be allocating more memory for the function to execute. You could go with the least memory if yours is a simple function. Memory range option varies from 128MB to 8GB as you can see below. You could leave the rest of the settings to default.

Google Cloud Function Memory Allocation Options

Step 3 : Cloud Function Code

Once you finish the configuration setup from the previous step, you will see an inline code editor. Here, we set our runtime to Python 3.7 and by default we will be provided main.py and requirements.txt files as you can see below.

You could add more files/scripts if necessary. The Entry Point handler is the function which resides in main.py and is run when triggered by an event. And requirements.txt has all the libraries and dependencies that needs to be installed to execute the function.

We have added model.py script to the code editor which has code to re-construct the model that it was trained on as we will be loading the model from its weights. utils.py contains the necessary functions like image/text preprocessing etc. generate_caption function is the entry point handler in the main.py file which takes in the input request and performs the inference on it.

Here are some ways to pass input image requests -

Encode images as base64 and pass b64 string to the cloud function.
Download images to Storage and pass the image path as request to the cloud function.

Base64 encoded image as input request

We will be passing base64 encoded image as input request in this section. Let us take a look at main.py script. The download function retrieves model weights and necessary files from your cloud storage bucket and downloads it to /tmp directory. It is the only directory cloud function allows us to write to.

Make sure to set your model as a global variable so it will be cached and reused during warm invocations. generate_caption takes in the input request as b64 encoded image and decodes it and saves it to the /tmp directory. load_and_prep_image loads the image from the /tmp directory to process the image for inference.

Once you are done with setting up the cloud function, you could deploy it by hitting the Deploy button at the bottom of the page. You could always go back and edit the function and Deploy it again if there are errors. It will take a while to deploy.

You can test the cloud function by hitting the ‘Testing’ tab. In this tab, you trigger the cloud function by passing the input request in the JSON format. The function’s output will be displayed in the Output section. You can find the function trigger URL in the ‘Trigger’ section.

Cloud Storage Bucket path as input request

generate_caption takes in the input image path (Google cloud storage path) as request and downloads the image from Google Cloud storage to /tmp directory. load_and_prep_image gets the image from /tmp directory to process it.

Step 3 : Deploy Streamlit app to Google App Engine

Create a project folder on your local machine and clone all the files required from here. Let’s take a look at app.py

We have utilised streamlit’s library to create headers, title, upload image to the app etc. You can add more widgets as necessary. In order to generate google app credential keys, please follow the below steps.

Navigate to IAM & Admin.
Go to Service Accounts and hit Create Service Account under that section.
Provide a name and add the role as AI Platform Developer and click Done to create the service account.
Click on the three dots icon under Actions and hit Manage Keys. This will take you to the Keys page.
Click on Add Keys to Create new key and select JSON as the recommended format.

Please make sure to keep this JSON file in your project folder and remember NOT to share this key with anyone.

Now, let’s get back to app.py, GC_URL is the trigger URL of our cloud function. file_uploader allows the user to add their own image and we save it to /tmp directory as test_file.jpg. We then upload the input image to Google Cloud Storage bucket.

Triggering Cloud Function

If the user hits on ‘Generate Caption’ (pred_button) button on our web app, it triggers the cloud function to output the caption of the image as the inference code resides in the cloud function.

Note : Remember to pass the image path (from Google cloud storage) as a JSON param when posting requests to the cloud function.

Run streamlit run app.py from your terminal which will open up the Streamlit web app from your browser and test the app locally.

Dockerize Streamlit App and test locally

Note : If you are already familiar with this step, feel free to jump ahead to next section below or you can check out my article here under Step 2.

We need a Dockerfile which contains instructions to build a container. Make sure to run the image locally and test it out before deploying it to the app engine.

Deploy Streamlit app to App Engine

We need app.yaml file which contains configuration settings for the app engine. For simplicity, we just have basic settings here. Our prediction code is on Google Cloud Function which requires more memory to perform inference and we have allocated 4GB when creating the function in Step 2. If we had our inference code on app.py, it is necessary to set up resources on app.yaml, depending upon your project. If yours is a simple model, you could leave it to default settings.

You can deploy the app using the command gcloud app deploy

This command pushes the docker image to the cloud and stores it in Google Container Registry and deploys the container on the app engine. This can take a while to run. Now, you can see the app on the destination url.

Congratulations!! You have now deployed your model on Google App Engine by triggering Google Cloud Function.

Additional Notes

Additionally, you could also setup a feedback loop/model retraining pipeline by logging in feedbacks from users.

Troubleshooting errors

Your client doesn’t have permission to access this server.

Go to your cloud function and hit Permissions tab and add allUsers. Select Role — Cloud Functions Invoker.

NOT_FOUND: Unable to retrieve P4SA: [service-1037200076078@gcp-gae-service.iam.gserviceaccount.com] from GAIA. Could be GAIA propagation delay or request from deleted apps.

gcloud app deploy command threw this error. Waited and retried the same command after sometime and it worked for me.

To debug your cloud function errors —