Evaluating EKM Performance Impact: Part 1 — Performance Test on GCP with Thales DPOD Service

Published in

Google Cloud - Community

6 min readNov 15, 2023

Database security often requires encryption, a process typically managed by Key Management Services (KMS). However, with rising concerns over security and sovereignty, external encryption keys have become an alternative. This study directly compares the performance impact of using conventional KMS keys against external encryption keys in database operations.

Test Objective

Our aim is to evaluate the performance implications of using external encryption keys on database operations. To achieve this, we will create two identical database instances within the same configuration parameters, with one significant variation: the encryption methodology. The first instance will employ conventional Key Management Service (KMS) keys, while the second will integrate external keys. Our goal is to discern any performance discrepancies between the two setups.

Tested Environments

We will deploy our testing framework using the open-source pgbench tool, executed from a Bastion host on RokyLinux. The virtual machine is provisioned on the Google Cloud Platform (GCP) in Paris (europe-west9Region) , configured with ( basic SKU n2-standard-4) 4 vCPUs, 16 GB of memory, and a 50 GB SSD. Connectivity to the Bastion host will be established via an Identity-Aware Proxy (IAP) tunnel.

For the External Key Management (EKM) system, we will be utilizing Thales Data Protection on Demand, witch is “a cloud-based platform providing a wide range of cloud HSM and key management services through a simple online marketplace”. It is important to note that this service does not allow for granular customization; keys are generated within Thales’s platform and then made available within Google Cloud Platform (GCP) through a specified URL, so they are only accessible over the internet.

KMS keys are created in the same region as Database and the Bastion Host

The target database is a PostgreSQL 15 instance, residing in the same region as the Bastion host. We’ll use a basic SKU (1db-n1-standard-4), featuring 4 vCPUs and 16 GB of RAM.

Test Procedure

We employed Terraform to orchestrate our test environment according to the characteristics outlined above. The scripts and resources are publicly available on GitHub, allowing anyone to replicate our EKM performance tests on GCP. Here’s how you can set it up:

Clone the repository:

git clone https://github.com/belgacem-io/ekm-performance-test.git

2. Set up environment variables by creating a file named `.env/gcp.env` with the following content (replace placeholders with actual values):


# GCP project where all resources will be created
export PROJECT_ID=xx-myproject-test-xxx
# GCP project where all keys are managed, can be the same as PROJECT_ID
export GCP_KMS_PROJECT_ID=xx-security-test-xxx
# Service account email used for creating resources                           
export GCP_IAC_SERVICE_ACCOUNT=xx@xx-myproject-test-xxx.iam.gserviceaccount.com        
# Default region where resources will be created
export GCP_REGION=xxxxx
# KMS Key URL used for encryption, can be empty                        
export GCP_KMS_KEY_URL=https://xxxxxxx

3. Load the environment variables:

source .env/gcp.env

4. Authenticate with GCP:

gcloud auth application-default login --no-launch-browser
gcloud config set project $PROJECT_ID

5. Provision the required resources using Terragrunt:

terragrunt --terragrunt-working-dir terragrunt/gcp-thales-dpod run-all apply

6. Connect to the Bastion host through IAP tunneling:

INSTANCE_NAME=$(gcloud --project=$PROJECT_ID compute instances list --format="value(name)")
gcloud --project=$PROJECT_ID compute ssh ${INSTANCE_NAME}

7. Install the necessary tools:

sudo bash /etc/test/run_perf_tests.sh --install

8. Execute pgbench tests on the database using KMS encryption, The test will simulate activity from 50 clients using 5 threads and runs for 10 minutes:

# This script will run the following pgbench command :
#. pgbench --log --protocol=extended --report-per-command --jobs=5 --client=50 --time=600 --host=xxx --username=xxx xxx

sudo bash /etc/test/run_perf_tests.sh --kms

9. Conduct pgbench tests on the database using EKM encryption, The test will simulate activity from 50 clients using 5 threads and runs for 10 minutes:

# This script will run the following pgbench command :
#. pgbench --log --protocol=extended --report-per-command --jobs=5 --client=50 --time=600 --host=xxx --username=xxx xxx

sudo bash /etc/test/run_perf_tests.sh --ekm

Our approach ensures that any performance variances observed can be attributed with reasonable certainty to the use of external keys versus conventional KMS keys.

Test Results

The benchmarking results from the pgbench tool offer an insightful comparison between databases using KMS encryption and those utilizing EKM encryption. Here’s an analysis of the test outcomes:

KMS Encryption Results

Total Transactions Processed: 1,689,707
Transactions Per Second: 2817.61 (excluding initial connection time)
Average Latency: 17.746 milliseconds
Initial Connection Time: 341.659 milliseconds
There were no failed transactions, indicating robust performance and reliability during the test period.

The latency for each statement was generally low, with the most time-consuming operation being the UPDATE on pgbench_branches at an average of 6.920 milliseconds. The transaction flow was consistent and efficient.

pgbench (15.5, server 15.4)
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 20
query mode: extended
number of clients: 50
number of threads: 5
maximum number of tries: 1
duration: 600 s
number of transactions actually processed: 1689707
number of failed transactions: 0 (0.000%)
latency average = 17.746 ms
initial connection time = 341.659 ms
tps = 2817.614514 (without initial connection time)
statement latencies in milliseconds and failures:
         0.001           0  \set aid random(1, 100000 * :scale)
         0.000           0  \set bid random(1, 1 * :scale)
         0.000           0  \set tid random(1, 10 * :scale)
         0.000           0  \set delta random(-5000, 5000)
         0.930           0  BEGIN;
         1.102           0  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
         1.088           0  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
         2.810           0  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
         6.920           0  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
         1.080           0  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
         3.809           0  END;

EKM Encryption Results

Total Transactions Processed: 1,564,075
Transactions Per Second: 2608.22 (excluding initial connection time)
Average Latency: 19.170 milliseconds
Initial Connection Time: 370.794 milliseconds
Similar to the KMS scenario, there were zero failed transactions.

pgbench (15.5, server 15.4)
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 20
query mode: extended
number of clients: 50
number of threads: 5
maximum number of tries: 1
duration: 600 s
number of transactions actually processed: 1564075
number of failed transactions: 0 (0.000%)
latency average = 19.170 ms
initial connection time = 370.794 ms
tps = 2608.218197 (without initial connection time)
statement latencies in milliseconds and failures:
         0.001           0  \set aid random(1, 100000 * :scale)
         0.000           0  \set bid random(1, 1 * :scale)
         0.000           0  \set tid random(1, 10 * :scale)
         0.000           0  \set delta random(-5000, 5000)
         0.968           0  BEGIN;
         1.162           0  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
         1.150           0  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
         3.028           0  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
         7.546           0  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
         1.128           0  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
         4.181           0  END;

In comparison to KMS, the EKM-encrypted database exhibited higher latencies for each statement, most notably with the UPDATE on pgbench_branches which took 7.546 milliseconds on average, indicating a slower processing time.

Comparative Analysis

When comparing both results, the database utilizing KMS encryption performed better with higher throughput and lower latency. Specifically, the KMS database handled approximately 125,632 (+ 7.5%) more transactions and achieved a higher TPS rate. Additionally, the average latency was 1.424 (- 8%) milliseconds less than that of the EKM-encrypted database. The initial connection time for KMS was also faster than EKM by about 29 milliseconds (- 8%).

Conclusion

The analysis indicates that KMS encryption offers better performance in terms of both latency and throughput (+ 8%). This could be due to the more integrated and possibly optimized nature of KMS within the same cloud service provider’s infrastructure, as opposed to the potentially more complex communication and encryption processes involved with an external key management system. It’s important to note that while KMS offers performance advantages, EKM may still be preferred in scenarios where external key management is a requirement due to policy or regulatory considerations.