Encryption using AWS KMS : CMK Support in CDE

Recently, Cloudera Data Engineering(CDE) solution has added the support of Customer Managed KMS key (CMK). Now, for those customers who have CMK configured in their Cloudera Data Platform (CDP) environment, CDE service will do CMK-based Encryption on data-at-rest for EFS, RDS, and Kubernetes secrets.

Earlier all this data was encrypted using the default KMS keys (AWS managed). Now the customers can also provide their own KMS key (CMK). With CMK, customers can have more control over the key.

Through this blog post, I want to share my experience and learnings around KMS keys, its access-control mechanisms.

Here I will cover below topics :

• What is encryption — Basic concepts
• Envelope encryption — Introduction and its benefits
• AWS KMS Keys — Basic understanding of KMS keys
• KMS Keys — Access control mechanism

What is encryption

Encryption is the process of converting data into a format that is unreadable or unintelligible to unauthorised parties. We use encryption to protect sensitive information like passwords, database secrets etc. to maintain its confidentiality, integrity and authenticity.

As part of the encryption process, the original data (passwords and other sensitive data) is known as plaintext. The plaintext is transformed using an encryption algorithm and an encryption key. The transformed data is encrypted data and called as cipher text.

The cipher-text can only be decrypted back into plaintext by authorised parties who possess the corresponding decryption key.

Encrypt data with a data key

Envelope encryption

The envelope encryption concept introduces one more key to encrypt/decrypt the plaintext encryption key. Through this way It adds one extra layer of protection around the data by separating the keys used for data encryption and key encryption.

So we have two keys :

• DEK — Data encryption Key
• KEK — Key encryption Key (Root key / Envelope key / Master key)

How it works

First the original data is encrypted using DEK, and then the DEK itself is encrypted using KEK, creating an envelope around the plain DEK. The encrypted DEK is also called the Envelope key.

Here Root key represents KEK (Envelope key)

The envelope key, along with the encrypted data is stored together.

During decryption, first the envelope key is decrypted using KEK, resulting in the plain DEK which is then used to decrypt the actual(encrypted) data.

Benefits of using envelope encryption

• Key Separation : It separates the keys for data encryption and key encryption. With it, If some attacker gets the encrypted data and DEK, they will still need KEK to decrypt the DEK to finally get the actual data decrypted.
• Key Rotation : It helps in easier Key rotation. The KEK can be easily changed without re-encrypting all the data. Only DEKs need to be re-encrypted using new KEK.

AWS KMS Keys

AWS KMS keys provide envelope encryption.

Definition from AWS :

An AWS KMS key is a logical representation of a cryptographic key. A KMS key contains metadata, such as the key ID, key spec, key usage, creation date, description, and key state. Most importantly, it contains a reference to the key material that is used when you perform cryptographic operations with the KMS key.

• Once the KMS key is created, then you can only use the key for different cryptographic operations (Encrypt, Decrypt etc), You cannot see the cryptographic material stored inside the key.
• You can also generate Data encryption keys (DEK) from KMS (using AWS API/CLI GenerateDataKey operation) and then you can encrypt the original data using this data key and encrypt this data key itself using KMS key which results in KEK (envelope key)
• During the decryption process, First the KEK (envelope key) will be decrypted using KMS key to get plaintext DEK and then using this DEK, the actual data will be decrypted.

Most of the AWS services are integrated with AWS KMS and can be encrypted using KMS keys.

For example — You can encrypt RDS, EFS using AWS KMS keys.

You can enable the encryption while creating EFS, RDS. If the encryption is enabled, then, by default the AWS managed KMS keys will be used for encryption. But you can define your own Customer Managed Key (CMK) for encryption.

AWS KMS has two types of Keys -

AWS Managed Keys

• AWS creates these by default for each AWS service which are integrated with KMS (For example — EFS, RDS)
• There are different keys for different AWS services but different instances of the same AWS service will use the same encryption key.

For example — Two different RDS instances will be using the same default KMS key (aws/rds).

Customer Managed keys (CMK)

• These are KMS keys created by users.
• Provides lot of flexibility. For example — You can rotate the keys
• Based on requirements, instead of using default AWS managed key, You can create different CMK keys for different instances of same AWS service. SO with CMK, two different instances of RDS can have different encryption key (which is not the case with default AWS managed keys)

How to control access to KMS key

Creation of the CMK is simple. You can create KMS key from AWS console or using AWS CLI :

aws kms create-key --description "my-kms-key"

Now the point comes how to control access to use it -

To control access to your KMS keys, you can use following mechanisms :

• Key Policy
• IAM Policy
• KMS Key Grants

Let’s discuss these one by one:

Key Policy

• It is the primary mechanism for controlling access to a KMS key.
• When we create a KMS key, a default key policy will be added.
• KMS key policy is a resource based access policy which tells about who(AWS principals — IAM user / role) can use the KMS key.
• You can use the key policy alone to control access.

Below is the default key policy:

{
  "Sid": "Enable IAM User Permissions",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::<account-id>:root"
   },
  "Action": "kms:*",
  "Resource": "*"
}

• It gives the AWS account that owns the KMS key full access to the KMS key.
• It allows IAM policies to use this key.
• Without this permission, IAM policies that allow access to the key are ineffective, although IAM policies that deny access to the key are still effective.

The principal in this key policy statement is the account principal which is represented by an ARN in this format: arn:aws:iam::<account-id>:root. When the principal in a key policy statement is the account principal, the policy statement doesn’t give any IAM principal permission to use the KMS key. Instead, it allows the account to use IAM policies to delegate the permissions specified in the policy statement.

Note — Instead of using account principal, You can also define a separate admin user in the key policy to have all the kms:* permissions, but still this default statement is important, because the admin IAM role can also be deleted (accidentally), So in that case If we don’t have account principal in the key policy then we will lose the admin control over the key and the key will become unmanageable. The account principal can be deleted only when the AWS account itself is deleted.

IAM Policy

• IAM policy is the standard way (identity-based) to do access-management in AWS.
• Unlike other aws resources, to allow access to any IAM role or user, the access should also be provided by KMS key policy as well.
• This means, Even If you have an IAM role which has the permission to use kms key actions but still these actions will not be allowed on the key unless we explicitly allow these actions from KMS key policy using account-principal or using this IAM role itself (to be more specific)

For example :

Let say we have a IAM role having below IAM policy :

Role arn — arn:aws:iam::111122223333:role/example-role

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Access KMS key full actions",
            "Effect": "Allow",
            "Action": [
                "kms:*"
            ],
            "Resource": "*"
        }    
   ]
}

To allow access to above IAM role, the KMS key policy must contain at least one of the two policy statements :

1. Using account-principle (see Default key policy)

{
    "Sid": "Enable IAM User Permissions",
    "Effect": "Allow",
    "Principal":
    {
        "AWS": "arn:aws:iam::111122223333:root"
    },
    "Action":
    [
        "kms:*"
    ],
    "Resource": "*"
}

2. Using specific IAM role (Key policy explicitly allowing access to IAM role)

{
    "Sid": "Allow KMS key access",
    "Effect": "Allow",
    "Principal":
    {
        "AWS": "arn:aws:iam::111122223333:role/example-role"
    },
    "Action":
    [
        "kms:*"
    ],
    "Resource": "*"
}

Note — Instead of kms:*, You should use specific actions based on your requirements.

KMS Key Grants

• Grants are temporary access provided to an AWS principal (user, role or service principals) to access and use the KMS key.
• These are used for temporary permissions because you can create a grant, use its permissions, and delete it without changing your original key policy or IAM policy.
• AWS services which are integrated with KMS (EFS, RDS…) commonly create and use grants to encrypt your data at rest using a given KMS key.
• The service creates a grant on behalf of a user/role in the account, uses its permissions, and retires the grant as soon as its task is complete.
  For example — Let say an EFS instance got created using an IAM role, named as example-role, then the example-role role must have kms:CreateGrant permission so that the EFS service can create grant to use the provided KMS key for encryption/decryption of data.

Let’s understand it with an example :

Suppose you want to create an EFS with encryption enabled to use given CMK key:

Assumed IAM role

"arn:aws:iam:us-west-2:111122223333:role/my-iam-role"

This IAM role has below permissions

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Access KMS key full actions",
            "Effect": "Allow",
            "Action": [
          "kms:*"
            ],
            "Resource": "*"
       },
       {
            "Sid": "EFS access",
            "Effect": "Allow",
            "Action": [
          "elasticfilesystem:*"
            ],
            "Resource": "*"
       }
  ]
}

CMK key

"arn:aws:kms:us-west-2:111122223333:key/my-kms-key"

CMK key policy

{
    "Sid": "Allow KMS key access",
    "Effect": "Allow",
    "Principal":
    {
        "AWS": "arn:aws:iam:us-west-2:111122223333:role/my-iam-role"
    },
    "Action":
    [
        "kms:*"
    ],
    "Resource": "*"
}, 

{
    "Sid": "Allow access to create and manage key grants",
    "Effect": "Allow",
    "Principal":
    {
        "AWS": "arn:aws:iam:us-west-2:111122223333:role/my-iam-role"
    },
    "Action":
    [
        "kms:CreateGrant",
        "kms:ListGrants",
        "kms:RevokeGrant"
    ],
    "Resource": "*",
    "Condition":
    {
        "Bool":
        {
            "kms:GrantIsForAWSResource": "true"
        }
    }
}

Note : The condition kms:GrantIsForAWSResource is to restrict that the grant should be generated for AWS resources only such as EFS, RDS etc. This condition is recommended to be there in KMS key policy around kms key grant operations, for security purposes, to protect grant creation.

To create EFS with encryption enabled, you can run following AWS CLI command (You can also create EFS using AWS console and pass the KMS ID):

aws efs create-file-system \
  --creation-token my-demo-efs \
  --performance-mode generalPurpose \
 --encrypted true \
  --throughput-mode bursting \
  --kms-key-id "arn:aws:kms:us-west-2:111122223333:key/my-kms-key"

Let’s understand which permissions will be used and where :

1. IAM role will be used to create EFS instance.
2. EFS instance will call KMS key to encrypt and decrypt the data using EFS service role (elasticfilesystem.us-west-2.amazon.com)
3. KMS key access will be allowed based on its key policy.

So, based on KMS key policy (mentioned above), Ideally, the EFS should not be able to access KMS key as the EFS service principal is not there in KMS key policy.

And that’s where the KMS key grants come in picture.

Internally, AWS EFS instance (using the current IAM role which already has kms:CreateGrant action allowed) will first create a grant for EFS service principal (elasticfilesystem.us-west-2.amazon.com) , so that EFS instance can encrypt/decrypt the data using the given KMS key.

Once the grant is generated for EFS service principal, then EFS instance can use KMS to encrypt/decrypt data without modifying the KMS key policy (And that’s the beauty of KMS key grants 🙂)

Note — Once the operation is done, EFS will Retire grant (i.e. deletes the grant)

To play around grants, you can run below commands to create and retire (delete) grants :

# aws kms create-grant \
  --key-id arn:aws:kms:<region>:<account-id>:key/my-kms-key \
  --grantee-principal "arn:aws:iam::<account-id>:role/my-iam-role" \
  --operations GenerateDataKey,Encrypt,Decrypt

# aws kms retire-grant --grant-token $token

Ref. Cloud Trail event log around create Grant by EFS

"sourceIPAddress": "elasticfilesystem.amazonaws.com",
"userAgent": "elasticfilesystem.amazonaws.com",

"requestParameters": {
        "constraints": {
            "encryptionContextEquals": {
                "aws:elasticfilesystem:filesystem:id": "fs-0d1122f1fb0122223"
            }
        },
        "keyId": "arn:aws:kms:us-west-2:111122223333:key/my-kms-key",
        "granteePrincipal": "elasticfilesystem.us-west-2.amazonaws.com",
        "operations": [
            "Encrypt",
      "Decrypt",
            "RetireGrant"
        ],
        "retiringPrincipal": "elasticfilesystem.us-west-2.amazonaws.com"
    },

"responseElements": {
        "grantId": "6d0477c57fbb84b4aa69044d45072bc2635ad6c163616d25e5a638e9edd7bc40",
        "keyId": "arn:aws:kms:us-west-2:111122223333:key/my-kms-key"
},

Referred Links :

- https://docs.cloudera.com/data-engineering/cloud/prereqs/topics/cde-aws-prereqs.html
- https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html
- https://docs.aws.amazon.com/kms/latest/developerguide/control-access.html
- https://docs.aws.amazon.com/kms/latest/developerguide/grants.html
- https://docs.aws.amazon.com/kms/latest/developerguide/service-integration.html