Google Cloud KMS based Service Accounts for Authentication and SignedURLs

salmaan rashid
Mar 2 · 8 min read

Tutorial on using a Google Cloud KMS key as a Service Account. KMS allows you to import an external RSA key and then invoke an API call to sign some arbitrary data using that key.

The ability to sign some data is the first step in oauth2 service account based flows (2LO) so if you can embed a key to KMS, you can derive a an access_token for that account or make it sign an GCS URL.

> Note: this flow is somewhat impractical at scale since each invocation to sign or generate an access token requires an API call. If the volume requirments is low, you consider using this procedure.

Note: this code is NOT supported by Google. caveat empotor

There are two ways to associate a Service Account with a KMS key:

Create a private key within KMS and then associate a Service Account with it.

or

Create a Service Account keypair; export the private key and import that key into KMS.

Once the Serivce Account private key is within KMS, you can do several things:

  1. Authenticate as that service account to a variety of GCP Services
  2. Generate a GCS SignedURL (or generally sign some data)

Ofcourse not matter what you do, you must have IAM access to the KMS key itself before you do anything. In the case where you’re authenticating against a GCP API, you could just have direct access to the target resource but if policies madate you need to use KMS based keys for some reason, you can use this procedure (note, if you just want to impersonate a service account itself to sign or authenticate, consider Impersonated TokenSource)

Core library implementation: cryto.Signer for KMS

The core library used in this sample is an implementation of the crypto.Signer interface in golang for KMS (again, that is on _my_ repo; its not officilly supported!). The Signer implementation allows developers to use higher-level golang constructs to do a variety of things that rely on cryptographic signing such as using net/http directly for mTLS. However, in our case, we will use the signing interface to generate a SignedURL and also use it within oauth2 TokenSource that relies on serive account signatures for authentication.

For reference, see:

GCP Authentication

KMS based Signer:

Setup

The steps below will setup two KMS keys: (1) one where you first generate a service account keypair and then import it into KMS and (2) one where you generate the a key within KMS and then associate it to an ServiceAccount.

In the first technique, the private key for a service account is exposed outside of KMS control (i.e, the private key at one point exists on disk). In the second, the private key never exists out of KMS control. The second option is significantly better since the chain of custoday of the key becomes irrelevant. However, you must ensure the association step to link the public certificate for the service account to the KMS key is carefully controlled.

Anyway, perform the following steps in the same shell (since we use several env-vars together)

first setup some environment variables

export PROJECT_ID=`gcloud config get-value core/project`export PROJECT_NUMBER=`gcloud projects describe $PROJECT_ID --format="value(projectNumber)"`export SERVICE_ACCOUNT_EMAIL=kms-svc-account@$PROJECT_ID.iam.gserviceaccount.com

create a service account (we will need this later)

gcloud iam service-accounts create kms-svc-account \
--display-name "KMS Service Account"
gcloud iam service-accounts describe $SERVICE_ACCOUNT_EMAIL

create a bucket and topic to test authentication:

export BUCKET_NAME=$PROJECT_ID-bucket
export TOPIC_NAME=$PROJECT_ID-topic
gsutil mb gs://$BUCKET_NAME
echo bar > foo.txt
gsutil cp foo.txt gs://$BUCKET_NAME/
gcloud pubsub topics create $TOPIC_NAME

Allow the service account access to gcs and pubsub

gcloud projects add-iam-policy-binding $PROJECT_ID  \ 
--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
--role=roles/storage.admin
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member=serviceAccount:$SERVICE_ACCOUNT_EMAIL \
--role=roles/pubsub.admin

create a keyring:

export LOCATION=us-central1
export KEYRING_NAME=mycacerts
export KEY_NAME=key1
gcloud kms keyrings create $KEYRING_NAME --location=$LOCATION

Import Service Account Private Key to KMS (SA -> KMS)

In this mode, you first generate a keypair for a Service Account, download it and then import it into KMS as described in Importing a key into Cloud KMS.

The specific steps to follow are:

a. Download .p12 key and convert to pem b. Create ImportJob c. Format pem key for import d. Import formatted key to kms via importJob e. Delete the .p12 and .pem files on disk

A) Create Service Account Key as .p12

$ gcloud iam service-accounts keys create svc_account.p12 \
--iam-account=$SERVICE_ACCOUNT_EMAIL \
--key-file-type=p12

for example:

Convert to PEM

$ openssl pkcs12 -in svc_account.p12  \
-nocerts -nodes \
-passin pass:notasecret | openssl rsa -out privkey.pem

B) Create ImportJob

Since we already cretae the keyring in the setup steps, we will just create the import job

export IMPORT_JOB=saimporter
export VERSION=1
$ gcloud beta kms import-jobs create $IMPORT_JOB \
--location $LOCATION \
--keyring $KEYRING_NAME \
--import-method rsa-oaep-3072-sha1-aes-256 \
--protection-level hsm
$ gcloud kms import-jobs describe $IMPORT_JOB \
--location $LOCATION \
--keyring $KEYRING_NAME

C) Format pem key for import

$ openssl pkcs8 -topk8 -nocrypt -inform PEM \
-outform DER -in privkey.pem \
-out formatted.pem

D) Import formatted via importJob

$ gcloud kms keys create $KEY_NAME \
--keyring=$KEYRING_NAME --purpose=asymmetric-signing \
--default-algorithm=rsa-sign-pkcs1-2048-sha256 \
--skip-initial-version-creation --location=$LOCATION \
--protection-level=hsm
$ gcloud kms keys versions import \
--import-job $IMPORT_JOB --location $LOCATION \
--keyring $KEYRING_NAME --key $KEY_NAME \
--algorithm rsa-sign-pkcs1-2048-sha256 \
--target-key-file formatted.pem

The service account key should now exists within KMS:

Finally, enable KMS key audit logs so we can see how its being used:

Test

Edit Test client main.go and update the the variables defined shown in the var() area. Note, keyId is optional

Run test client

go run main.go

You should see the output sequence:

a) A signed URL

$ go run main.go 2020/01/06 16:52:38 https://storage.googleapis.com/your_project/foo.txt?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=kms-svc-account%40yourproject.iam.gserviceaccount.com%2F20200107%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20200107T005237Z&X-Goog-Expires=599&X-Goog-Signature=27redacted&X-Goog-SignedHeaders=host

b) Response from an HTTP GET for that signedURL.

In our case its the content of the file we uploaded earlier as well as a 200 OK (which means the signedURL worked)

2020/01/06 16:52:38 SignedURL Response :
bar
2020/01/06 16:52:40 Response: 200 OK

c) List of the pubsub topics for this project

2020/01/06 16:52:41 Topic: sd-test-246101-topic

d) List of the buckets on this project

2020/01/06 16:52:42 sd-test-246101-bucket

Finally, since we enabled audit logging, you should see the KMS API calls that got invoked.

In this case its two invocations: one for the SignedURL and one for the other cloud services apis:

Import KMS Public Certificate as Service Account (KMS -> SA)

The follwoing procedure generates a key in KMS and associates its public certificte with a given ServiceAccount. This procedure is basically describe here:

a. Create a keyring (if you haven’t done so already)

b. Create a key with KMS keymaterial/version

c. Generate x509 certificate for KMS key

d. Create ServiceAccount (if you havent done so already)

e. Associate x509 certificate with ServiceAccount

Jumping straight to

B) Create a key with KMS keymaterial/version

export KEY2_NAME=key2-importedgcloud kms keys create $KEY2_NAME \
--keyring=$KEYRING_NAME --purpose=asymmetric-signing \
--default-algorithm=rsa-sign-pkcs1-2048-sha256 \
--location=$LOCATION

note, unlike the key that we imported, this does not require HSM (i.,e you can omit --protection-level=hsm)

C) Generate x509 certificate for KMS key

ServiceAccount import requires a the public x509 certificate but KMS does not surface an API for x509 but rather the public .pem format for a kms key is all that is currently provided (see cryptoKeyVersions.getPublicKey)

However, since we’ve setup a crypto.Singer for cloud KMS, we can use it to genreate an x509 certificate pretty easily.

Download certgen.go from the crypto.Singer implementation:

wget https://raw.githubusercontent.com/salrashid123/signer/master/certgen/certgen.go

Edit the certgen and specify the variables you used for your project. Note the keyname here can be key2-imported

r, err := salkms.NewKMSCrypto(&salkms.KMS{
ProjectId: "yourproject",
LocationId: "us-central1",
KeyRing: "mycacerts",
Key: "key2-imported",
KeyVersion: "1",
})

Generte the x509 cert:

$ go run certgen.go --cn=$SERVICE_ACCOUNT_EMAIL
2020/01/06 17:17:27 Creating public x509
2020/01/06 17:17:28 wrote cert.pem

Note the certificate is x509 (the cn doens’t atter but i’ve set it to the service account name)

$ openssl x509 -in cert.pem -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
f0:6e:7b:cf:2c:72:0d:8d:f9:16:38:61:ec:1e:a9:2d
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, ST = California, L = Mountain View, O = Acme Co, OU = Enterprise, CN = kms-svc-account@sd-test-246101.iam.gserviceaccount.com
Subject: C = US, ST = California, L = Mountain View, O = Acme Co, OU = Enterprise, CN = kms-svc-account@sd-test-246101.iam.gserviceaccount.com

D) Associate x509 certificate with ServiceAccount

The final step here is to upload and associate the public key with the service account we already created:

$ gcloud alpha iam service-accounts keys upload cert.pem  --iam-account $SERVICE_ACCOUNT_EMAIL

You should see a new KEY_ID suddenly show up. In my case it was:

$ gcloud iam service-accounts keys list --iam-account=$SERVICE_ACCOUNT_EMAIL
KEY_ID CREATED_AT EXPIRES_AT
ce4ceffd5f9c8b399df9bf7b5c13327dab65f180 2020-01-07T00:11:07Z 9999-12-31T23:59:59Z
db8f0a5af9cf3bd211f4936ab7350788d4c774d8 2020-01-07T01:17:27Z 2021-01-06T01:17:27Z <<<<<<<<<
1f1a216c7e08119926144ad443e6a8e3ec5b9c59 2020-01-07T00:05:47Z 2022-01-13T10:00:31Z

note, there is an org policy that allows you to specify which Certificate Authority Issuers are allowed in this step (constraints/iam.allowedPublicCertificateTrustedRootCA )…however, as of 3/4/10, that takes the _string form_ of the issuer as a list constraint (which isn’t useful; it needs to digitally verify….

Test

Edit Test client main.go and update the the variables defined shown in the var() area. Note, keyId is optional. Remember to update the keyName to key2-imported or whatever you setup earlier.

Run test client

go run main.go

The output should be similar to the first procedure.

woooo!!!

TPM and Yubikey based SignedURL and GCPAuthentication

You can redo the same procedure using a Trusted Platform Module (TPM) or even a Yubikey too! I may add in an article about that shortly but for now, see:

Google Cloud - Community

A collection of technical articles published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

salmaan rashid

Written by

Google Cloud - Community

A collection of technical articles published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

More From Medium

More from Google Cloud - Community

More from Google Cloud - Community

More from Google Cloud - Community

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade