Cloud CDN — Best Practices
Using the Akamai CDN and a Private Google Storage Bucket
It is fast becoming common practice to host static objects on an elastic storage system such as Google Cloud Storage and fronting the storage repository with a CDN platform such as Akamai to provide global scale and performance to end users. Content Providers can leverage the advanced functionality that the Akamai CDN configuration platform can offer, and combine it with the global scale, reliability, and unit economics that Google Storage can provide.
By default, uploaded assets to the Google Cloud Storage service are encrypted and private; however, content can be made accessible when presented with an authorization token. The challenge is how to dynamically generate the ephemeral access token. This article discusses the process by which you can configure your Akamai service to access a private Google Cloud Storage bucket.
Challenge
The Akamai CDN platform needs to pull content from a publicly accessible HTTP web server. In the case of the Google Cloud Storage service, this would necessitate making the storage bucket assets publicly available, which is generally not a best practice as it can expose Content Provider content that wasn’t intended to be made public. The difficulty here is how can Akamai’s CDN platform access content that isn’t publicly available?
Solution
Fortunately for us, the Akamai’s CDN platform has the ability to dynamically generate an access token leveraging the Origin Characteristics behavior functionality within the Akamai CDN configuration, which will allow temporary access to the Cloud Storage private bucket. There are three basic steps to follow in order to configure the Akamai CDN to access a private Google Cloud Storage resource.
- Create a private Cloud Storage bucket
- Create an IAM service account with Read Only permissions and generate an HMAC token for API access
- Modify Akamai configuration with Origin Characteristics behavior
1. Setup Google Storage Bucket
For the purposes of this blog post, I won’t go into setting up a Google Storage bucket. You can consult with the online Google Cloud documentation for more information on setting up a storage bucket. By default, all uploaded content is encrypted at rest and is made private. For the purpose of this blog post, I created a private Cloud Storage bucket called “cloudcdn-private-bucket”.
2. Create Read Only Service Account and API Key
The next step in the process is to create a user role that will allow the Akamai service to read from the Cloud Storage bucket. As a best practice, you want to create a separate service role that has limited scope to only read from the storage service (and specific bucket) to ensure that if the credentials leak, the blast radius is contained only to the storage service and nothing else within your Google Cloud project. In my example, I am going to set the scope to a particular storage bucket.
Cloud Console
- Log into the Google Cloud console.
- Navigate to your Storage account: Home > Storage > Browser. The resulting page will list your available storage buckets.
- Click on the storage bucket your want to grant access to. In my example, I am going to use a bucket called “cloudcdn-private-bucket”
In my bucket, I have uploaded an image called dog.jpeg. Just to confirm that the bucket is private, I ran a curl statement to the object on Cloud Storage and I get redirected to a login page as the asset is private and doesn’t allow anonymous user access.
4. Verify that the Cloud Storage bucket is closed by clicking on the PERMISSION sub-navigation tab in the bucket. You should see messaging indicating that the storage bucket is private.
5. Navigate to the Settings in the Storage section: Home > Storage > Settings. Click on the INTEROPERABILITY sub-navigation tab.
6. Under the Service account HMAC section, click on the button “+ CREATE A KEY FOR A SERVICE ACCOUNT. This is where I will create the API access key that I will use with the Akamai Origin Characteristics behavior to generate the ephemeral access token.
7. Click the “CREATE NEW ACCOUNT” button.
8. Create a service account name. I am using “akamai-read-only” as the service account name. Click the CREATE button.
9. Select a role for the service account. From the pull down menu, select the “Storage Object Viewer” role. As a shortcut, you can type “storage object” in the filter section to quickly locate the permission.
10. [Optional] To restrict the role to the storage bucket, click on the Add Condition link. Add a title to the conditional rule name. I am using “storage bucket name”. Select Condition = “Name”, Operator = “is”, where the value attribute is the name of your Storage bucket that you are trying to grant access to. In my example, I am using “cloudcdn-private-bucket”. Click the SAVE button.
NOTE: If you do not restrict the read only permission to a specific Cloud Storage bucket, the new service account will have the ability to read objects from all of the storage buckets in your project.
11. Next, click the CONTINUE button to save the service permission for the service account.
12. Click the DONE button. In my example, I am not granting the read only rights to an individual user role- just granting programmatic access to the storage service.
13. After you click DONE from the previous step, the console will generate your Access key and Secret value that I will need for configuring my Akamai configuration.
NOTE: You will need to write down the secret value as that value will only be displayed once. If you don’t write down the Secret value, you will need to create a new HMAC key value.
You can verify that the service account permission is assigned to your private storage bucket by navigating to Home > Storage > Browser. Select the bucket you are trying to grant access to and then click on the PERMISSIONS sub-navigation tab. This is similar to the Step 4 task above. Scroll to the bottom of the page where it should list the newly created service account.
GCLOUD Commands
Using the Google command line tools, you can also generate the HMAC key values needed to setup your service account permissions. For the sake of brevity, I am not going to illustrate how to setup a Cloud Storage bucket via gcloud commands and just focus on setting up the access permissions.
- Create the read only service account
a. NAME: the internal name of the new service account used to generate an IAM_ACCOUNT
b. Description: a user created explanation of the service account
c. Display Name: the name value used to list the account
NOTE: make sure you add “ “ marks around the description and display name values if you have spaces in the description.
Format:
gcloud iam service-accounts create NAME [--description=DESCRIPTION] [--display-name=DISPLAY_NAME]Example:
gcloud iam service-accounts create akam-read-only --description "read only access to gcs" --display-name "akamai read only access"2. Get the service account email value generated by the previous command. I am filtering the list of service accounts returned by limiting the results to service accounts with the words “akam” in the name value. This will limit your query results so you don’t have to sift through the entire project list of service accounts.
gcloud iam service-accounts list --filter "akam"Output:
3. Using the service account email, generate the HMAC key
Format:
gsutil hmac create [service_account_email]Example:
gsutil hmac create akam-read-only@playground.iam.gserviceaccount.comOutput:
+------------+----------------------------------------+
| Access ID: | GOOGXXXXXXXXXXDEBMO7OYU3VWBVLYNJ5QQKP7 |
+------------+----------------------------------------+
| Secret: | G+GrwQJAB^*YEC5ppStIKcxsD9TZjTnx |
+------------+----------------------------------------+3. Configure Akamai Origin
Within your Akamai configuration, you need to modify your settings using the following Behaviors:
- Origin Server
- Origin Base Path
- Origin Characteristics
The origin server value will be configured to source origin content from your private Google Cloud Storage bucket. The origin base path will be the name of your bucket. Lastly, the Origin Characteristics function will dynamically generate the ephemeral access token that will allow you to read objects from the Cloud Storage bucket.
Origin Server
The origin server behavior is the configuration component that tells the Akamai edge servers where the origin server is located. In particular, you need to modify the following settings:
- Origin Server Hostname: “storage.googleapis.com”
- Forward Host Header: select Origin Hostname
- Cache Key Hostname: select Origin Hostname
- Verification Settings: Use Platform Settings
- Use SNI TLS Extension: YES
Origin Base Path
The Origin Base Path tells the Akamai edge server to add a path value to the origin request. I will use this functionality to forward the Google Cloud bucket name value to the origin request. Without the Origin Base Path, Google will not know which bucket the request is trying to access.
The Origin Base Path component is not included in the default Akamai configuration setup. You will need to click on “Add Behavior” and select Origin Base Path. Normally, this will be added into the Default Rule section, but depending on the complexity of your configuration, you may need to add the module elsewhere in your configuration.
- Base Path: /cloudcdn-private-bucket/
Make sure that your Base Path value starts and ends with a backward slash (/).
NOTE: If you are using multiple Google Cloud Storage buckets, you can set up the Origin Server and Origin Characteristics in the default path and then create a matching rule on path or file where you modulate the Origin Base Path behavior on the match. The origin server and characteristic behavior will carry forward and just change the forward path value.
Origin Characteristics
The Origin Characteristics behavior is really where the magic happens. In this component, I will specify the Access ID and Secret that will be used to dynamically generate the access token value needed to authenticate the Akamai edge server request on the Cloud Storage platform.
LIke the Origin Base Path component, the Origin Characteristics module is not included in the default Akamai configuration setup. In the Default Rule, click Add Behavior and select Origin Characteristics component.
There are four configuration elements that need to be adjusted:
- Origin Location: select the region where your Cloud Storage bucket is located. In my example, I am using North America
- Authentication Method: select Interoperability Google Cloud Storage
- Access ID: this can be confusing in that Akamai’s configuration portal calls the attribute Access ID but in the Google console, the attribute is called Access Key. You will want to take the Access Key value you generated from the prior step and insert that value here.
- Secret: the Secret value is the one time value that was shown to you when you created the service account’s access credentials. Insert the cryptographic nonce value here.
Save your Akamai configuration and subsequently activate the configuration to the production network.
Testing your setup
To verify your setup, you can insert the URL in your web browser to see if the object renders for you.
Alternatively, you can run a curl command to check the header for a HTTP 200 response.
Troubleshooting Tips
Some common troubleshooting issues that may crop up include:
- Use HTTPS: the Google API hostname will only respond to requests using HTTPS. If your Akamai configuration uses the HTTP protocol, you may need to force upgrade the backhand pass protocol to the origin to only use HTTPS.
- Verify SNI flag in your Akamai Origin Server setup. Make sure that the Origin Server behavior utilizes the SNI flag
- Verify the Forward Host header value. The value should be set to the origin hostname value and not the Incoming Hostheader Value (which is the default setting).
- Incorrect Access ID/Access Key: be sure that you are using the correct credentials for the Cloud Storage bucket. Verify that the HMAC credentials on your private Cloud Storage bucket are active.
- Don’t have permissions to create IAM service account or assign role permissions to a service account.
If you have questions about Google Cloud and Cloud CDN, contact your Google Cloud Sales team or reach out to me via the Google Cloud Community Slack channel and post a note in the #cloud-cdn channel.
