Secure Cloud Run, Cloud Functions and App Engine with API Keys and Cloud Endpoint

Security is one of the current major challenge and cloud providers offer different solutions for enforcing the security. Google Cloud proposes to secure the product with IAM roles and permission applied on accounts; technical (service-account) or personal (email, groups or domain wide).

Thereby, when you deploy in private mode Cloud Run or Cloud Function, or when you use App Engine with IAP, bearer authorization token is required for authentication and for checking the allowed roles and permissions on the authenticated account.

OAuth 2.0 is one of the best trade-off between lightweight and simplicity to use (add token in request header) and an acceptable security risk (the token life duration is short and limits the risk of man-in-the-middle attack).

However, about once a month, I read this kind of question on Stack Overflow:

Can I reach Cloud Run with API Key?

How I can secure my function with API Key?

If API Keys aren’t proposed for securing API endpoint, it’s because of its higher security risk. Indeed, the API key hasn’t a short life duration and its rotation implies a synchronisation between the client and the server. By the way, this long life duration increases the risk of man-in-the-middle attack and it’s not recommended to use an API key for securing the API access.

However, for some use cases, an API key stays a balanced security solution compare to “no security” or a basic HTTP authentication. But, as said before, Google Cloud doesn’t support natively this type of authentication. By the way, we have to use an additional layer: Cloud Endpoint, also known as Extensible Service Proxy (ESP) which is now an open source product.

Cloud Endpoint

Cloud Endpoint exists for many years with App Engine, but recently, Google Cloud has decided to extract the service from App Engine and to let anyone leverage of it.

Several deployment mode are possible, and I chose to use the Cloud Run mode. Despite the description of the tutorial, you don’t need to have several projects and my deployment example is a bit easier and try to avoid all the common pitfalls.

Start by creating a service-account for the Cloud Run service which will host Endpoint service (I will name it Cloud Run Endpoint gateway), and by allowing it the required role for deploying Endpoint container, create a service-account with the correct role (change <PROJECT_ID> with your project ID)

gcloud beta iam service-accounts create endpoint-id
gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --role=roles/servicemanagement.configEditor --member \
  serviceAccount:endpoint-id@<PROJECT_ID>.iam.gserviceaccount.com

And deploy Endpoint container on Cloud Run (I named the service endpoint)

gcloud beta run deploy endpoint \
 --image="gcr.io/endpoints-release/endpoint-runtime-serverless:1" \
 --allow-unauthenticated --region us-central1 --platform managed \
 --service-account \
 endpoint-id@<PROJECT_ID>.iam.gserviceaccount.com

Get the URL provided with the form https://endpoint-<hash>-uc.a.run.app. We will use it later, without thehttps:// as the ENDPOINTS_SERVICE_NAME

Now, your Cloud Run Endpoint gateway is running. We have to deploy the secured apps and to define the routes

Deploy the secured services

For testing Cloud Endpoint in various situation, let’s deploy services on Cloud Run (private mode), Cloud Functions (private mode) and App Engine (protected by IAP)

Source code

Before starting to deploy services, open Cloud Shell and clone the GitHub repository

git clone https://github.com/guillaumeblaquiere/apikey-endpoint.git
cd apikey-endpoint

The rest of the deployment will be done in this directory apikey-endpoint

You can have a look of the Go code. It’s a simple webserver with /hello GET API which reply the name if provided in query parameter, and the platform on which it runs (the platform is provided in environment variable)

Cloud Run

By default, Cloud Run is deployed in private mode by default, this means that unauthenticated user, or authenticated user with the role roles/run.invoker, can’t invoke the service (or an another role with the required permissions)

For deploying the service, there is 2 steps. First, build the container (change <PROJECT_ID> with your project ID).

gcloud builds submit --tag gcr.io/<PROJECT_ID>/apikey-endpoint

Then deploy the container in private mode: — no-allow-unauthenticated

gcloud beta run deploy apikey-endpoint --no-allow-unauthenticated \
    --image gcr.io/<PROJECT_ID>/apikey-endpoint \
    --region us-central1 --platform managed

Keep somewhere the provided URL. We will need it later. You try to perform a curl on it for validating that, without authentication, the service isn’t accessible

Cloud Function

By default, Cloud Functions are also deployed in private mode by default, this means that unauthenticated user, or authenticated user with the role roles/cloudfunctions.invoker, can’t invoke the function (or an another role with the required permissions).

The deployment takes only the files in the function source directory.

gcloud beta functions deploy apikey-endpoint --trigger-http \
   --runtime go112 --source function --entry-point HelloWorld \
   --region us-central1 --no-allow-unauthenticated \
   --set-env-vars=ENV="Cloud Functions"

Keep somewhere the provided URL. We will need it later. You try to perform a curl on it for validating that, without authentication, the service isn’t accessible

App Engine Standard

App Engine is public by default. However, Google Cloud has added a security layer named Identity-Aware Proxy (or IAP) which doesn’t allow to reach the service the unauthenticated users, or authenticated users without the role roles/iap.httpsResourceAccessor (or an another role with the required permissions).

There is 2 steps. First, deploy App Engine Standard (activate the App Engine API if required. Choose wisely your zone, you can’t change later!)

gcloud app deploy

Keep somewhere the provided URL. We will need it later.

And then, activate IAP on App Engine. There is no command line for this, but you can do it through the GUI.

Go to Security and select Identity-Aware Proxy. Fill in the consent page with your application name and go back on the previous tab. Here, put the slider to the right (ignore the Error flag) and validate the warning message by clicking onTurn on.

You try to perform a curl on the for validating that, without authentication, the service isn’t accessible.

Link Cloud Endpoint with the services

Now, we have to declare to Cloud Endpoint service where and how to reach the services. For this, you have to define a YAML file in OpenAPI v2.0 format. Some tools exists for this and for accelerating the development.

Look at the file endpoint.yaml. You can define a list of paths entry-point and the backend to reach.

Start by creating the file header:

swagger: '2.0'
info:
  title: Cloud Endpoints with API Keys
  description: Sample API on Cloud Endpoints with a Cloud Run, Cloud Function and App Engine with IAP backend
  version: 1.0.0
host: endpoint-<hash>-uc.a.run.app
schemes:
  - https

The host is the value of ENDPOINTS_SERVICE_NAME of your Cloud Run Endpoint gateway . The title is the name of your API in the API & Services section.

Cloud Run

For Cloud Run, it’s recommended to use the APPEND_PATH_TO_ADDRESS. This param configure Cloud Endpoint for copying the path /hello to the end to the address and do an “host rewriting”.

/hello:
  get:
    summary: Greet a user from Cloud Run
    operationId: hello_cloud_run
    x-google-backend:
      address: https://apikey-endpoint-<hash>-uc.a.run.app
      path_translation: APPEND_PATH_TO_ADDRESS

App Engine

For App Engine, we could also use an “host rewriting”. But the path /hello will be in double. Thereby, we don’t specify any path_translation for performing URL rewriting (change <PROJECT_ID> with your project ID)

/hello-gae:
  get:
    summary: Greet a user from App Engine
    operationId: hello_gae
    x-google-backend:
      address: https://apikey-endpoint-dot-<PROJECT_ID>.appspot.com/hello

Cloud Functions

For Cloud Function, and for keeping a consistency with the other names, we also don’t specify any path_translation for performing URL rewriting (change <PROJECT_ID> with your project ID)

/hello-gcf:
  get:
    summary: Greet a user from Function
    operationId: hello_cloud_function
    x-google-backend:
      address: https://us-central1-<PROJECT_ID>.cloudfunctions.net/apikey-endpoint

Work with API Keys

The forward is done, but there is no security for now. There is several action to do for activing and using the API Keys for authentication.

Add API Key security in Cloud Endpoint

The API Key security is to add globally or on each path like in my example.

security:
    - api_key: []

For tests, you can replace this by x-google-allow: all for disabling the security.

And add the security definition at the end of the file

securityDefinitions:
  api_key:
    type: "apiKey"
    name: "key"
    in: "query"

Deploy Cloud Endpoint Service

Now, the endpoint.yaml is fully defined, let’s go to deploy it on Cloud Endpoint service

gcloud endpoints services deploy endpoint.yaml

Now the Cloud Endpoint service exists. For indicating this to the Cloud Run Endpoint gateway, we have to update it with an environment variable that specify this name with the value ofENDPOINTS_SERVICE_NAME .

gcloud beta run services update endpoint \
--set-env-vars ENDPOINTS_SERVICE_NAME=endpoint-<hash>-uc.a.run.app \
--region us-central1 --platform managed

Activate your API

By deploying the config on Cloud Endpoint, a new API service is created. Now, you have to activate it on your project. You can do this by command line:

gcloud services enable endpoint-<hash>-uc.a.run.app

The API service name is the name of your Cloud Run Endpoint gateway ENDPOINTS_SERVICE_NAME

You can also activate the service through the GUI. Go to Google Cloud console, go to API & Services and select Library. Then, search for your API name (Cloud Endpoints with API Keys if you don’t change the title in the endpoint.yaml file) and activate it.

Create the API Key

For reaching the service, you have to use an API Key. For this you have to create one in Google Cloud console.

Go to API & Services and select Credentials. Click on Create credentials and select API Key.

Because of the low level of security of an API Key, the best practice is to restrict the key.

Edit the key (click on the pencil), under API restrictions, click on Restrict key and, in the drop down list, only check your API Name (Cloud Endpoints with API Keys if you didn’t change it in the title in the endpoint.yaml file)

Keep the key value of the created and authorized API Key, we will use it later.

Cloud Run Endpoint gateway authorizations

Let’s summarize what we have.

• We have 3 services on Cloud Run, Cloud Functions and App Engine. Each of these services are secured and an IAM role is required to access them.
• We also have a Cloud Run Endpoint gateway linked with Cloud Endpoint service. On this service, the required configuration for routing requests to each service has been deployed and it requires a valid API Key in each path entry-point.

But, how the Cloud Run Endpoint gateway can reach each secured service?

For this, we have to grant the Cloud Run Endpoint gateway service-account with the required authorization

Cloud Run

As describe previously, a private Cloud Run service can be reached by authenticated user with roles/run.invoker. So, let’s grant this role to the Cloud Run Endpoint gateway service-account.

gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --role=roles/run.invoker --member \
  serviceAccount:endpoint-id@<PROJECT_ID>.iam.gserviceaccount.com

Cloud Functions

Like for Cloud Run, and as described previously, a private Cloud Functions service can be reached by authenticated user with roles/cloudfunctions.invoker. So, let’s grant this role to the Cloud Run Endpoint gateway service-account (change <PROJECT_ID> with your project ID).

gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --role=roles/cloudfunctions.invoker --member \
  serviceAccount:endpoint-id@<PROJECT_ID>.iam.gserviceaccount.com

App Engine

App Engine secured by IAP is slightly different. Of course, the role roles/iap.httpsResourceAccessor has been to granted to the Cloud Run Endpoint gateway service-account (change <PROJECT_ID> with your project ID).

gcloud projects add-iam-policy-binding <PROJECT_ID> \
   --role=roles/iap.httpsResourceAccessor --member \ 
   serviceAccount:endpoint-id@<PROJECT_ID>.iam.gserviceaccount.com

You can also perform this authorization through the GUI

• Go to Security and select Identity-Aware Proxy.
• Select your App Engine and, on the right panel (click on Show info panel if not present)
• Click on Add Member.
• Paste your endpoint-id@<PROJECT_ID>.iam.gserviceaccount.com email account and select the role Cloud-IAP -> IAP-secured Web App User

In addition, IAP also check the correct audience of the JWT token. For this, we have to update the endpoint.yaml file for indicating the correct audience to Cloud Endpoint service when the routing is performed. (change <PROJECT_ID> with your project ID, and set the correct jwt_audience)

x-google-backend:
  address: https://apikey-endpoint-dot-<PROJECT_ID>.appspot.com/hello
  jwt_audience: <PROJECT_NUMBER>-<HASH>.apps.googleusercontent.com

The jwt_audience is the IAP client ID. You can find it by going to API & Services and select Credentials. Here, look at the OAuth 2.0 client IDs and look for IAP-App-Engine-app line and copy the Client ID

And redeploy the endpoint.yaml file to Cloud Endpoint service

gcloud endpoints services deploy endpoint.yaml

Validate Cloud Endpoint routes

Finally, all the set up is done. Take back your key value of your API Key and try to request the Cloud Run Endpoint gateway URL and validate that the routes work as expected

curl https://endpoint-<hash>-uc.a.run.app/hello?key=<API KEY>
curl https://endpoint-<hash>-uc.a.run.app/hello-gcf?key=<API KEY>
curl https://endpoint-<hash>-uc.a.run.app/hello-gae?key=<API KEY>

Cloud Endpoint for API Keys and more!

Even if the API Keys aren’t natively supported by Google Cloud services, for security reason, and even if this authentication mode isn’t recommended, if your use cases require it, it’s possible to address them with a serverless product: Cloud Endpoint.

I just presented a small part of Cloud Endpoint capabilities and I used it as API Gateway for abstracting the authentication and for the routing to different services. And you aren’t limited to Google Cloud services APIs. You can also use it to route your requests on-premise or on other Cloud Provider.

But, Cloud Endpoint can do much more. You can;

• Set up a custom domain.
• Deploy it on a VM, on Cloud Function, on GKE and Kubernetes.
• You can define quotas and rate limits.
• Integrate other authentication mode like Auth0 or FirebaseAuth (and it’s Google Cloud declination Cloud Identity Platform).
• Share a developer portal where your API is described and where the developers can test it.
• Have API versioning, restore previous API version and see the differences between versions.
• …

If you look for more advanced features, like billing and monitoring, Apigee is a better solution.

But, if you look for a powerful and free API gateway solution, Cloud Endpoint is for you. Let’s try it!