Cross account S3 object copying with KMS encrypted buckets

Photo by Markus Spiske on Unsplash

Sometimes, when your project is too large to handle everything in one AWS account or when you maintain multiple environments in different AWS accounts and you need the data to flow between accounts, it becomes a headache to do all that with reconfiguring the IAM roles and S3 bucket policies to move/copy objects from one bucket to another in different accounts and still facing AccessDenied errors.

One such scenario is when you try to copy objects between s3 buckets across multiple AWS accounts.

I have divided this blog into 2 sections, one where you are using default S3 encryption to encrypt the objects and another where you are using a KMS customer-managed key (CMK) to encrypt the objects.

Step 1: Create an IAM policy like the one below, replace the source and destination bucket names.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::SOURCE_BUCKET_NAME",
                "arn:aws:s3:::SOURCE_BUCKET_NAME/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::DESTINATION_BUCKET_NAME",
                "arn:aws:s3:::DESTINATION_BUCKET_NAME/*"
            ]
        }
    ]
}

Step 2: Attach the above policy to the IAM user or role that is doing the copy object operation.

Step 3: Change the Object ownership to Bucket owner preferred in the destination bucket.

Normally when you copy a file from one bucket to another in different AWS accounts, the owner of the copied object in the destination bucket will still be the source account. To change this behavior you need to configure this setting in the destination bucket permissions.

Step 4: Your destination bucket needs to have a bucket policy that gives access to the role you created in the source account. In the policy below, change the role ARN in the policy principal and bucket name in the resource list and attach it to your bucket.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "Cross account copy object S3 bucket policy",
    "Action": [
      "s3:ListBucket",
      "s3:PutObject",
      "s3:PutObjectAcl"
    ],
    "Effect": "Allow",
    "Resource": [
      "arn:aws:s3:::DESTINATION_BUCKET_NAME",
      "arn:aws:s3:::DESTINATION_BUCKET_NAME/*"
    ],
    "Principal": {
      "AWS": [
        "arn:aws:iam::123456789:user/username",
        "arn:aws:iam::123456789:role/role-name"
      ]
    }
 }]
}

Step 5: You need to specify object ACL as bucket-owner-full-control when you are doing the copy operation.

If you are using boto3:

s3 = boto3.client('s3')
s3.copy_object(Bucket = 'DESTINATION_BUCKET_NAME', 
               Key = 'destination_object.txt',
               CopySource={
                   'Bucket': 'SOURCE_BUCKET_NAME',
                   'Key': 'source_object.txt'
               },
               ACL = 'bucket-owner-full-control'
)

If using AWS CLI:

aws s3 cp s3://SOURCE_BUCKET_NAME/source_object.txt s3://DESTINATION_BUCKET_NAME/destination_object.txt --acl bucket-owner-full-control

When Objects are KMS Encrypted

Everything you have done above will work fine if you are using default s3 encryption in both the source and destination bucket. But, if you are using KMS Customer Managed keys for object encryption in any of the source or destination buckets, the IAM role/user which is being used to copy objects needs to have access to the KMS keys.

Note: I am assuming that the IAM user/role is in the Source Account.

If only source bucket objects are KMS encrypted:

The IAM user/role needs to have permission to do the following actions on KMS.

{
   "Sid": "Allow KMS actions",
   "Action": [
      "kms:Decrypt",
      "kms:DescribeKey",
      "kms:GenerateDataKey",
   ],
   "Effect": "Allow",
   "Resource": "your-kms-key-arn"
}

If only destination bucket objects are KMS encrypted:

The KMS key policy in the destination account should allow the IAM user/role in the source account the following actions:

{
  "Sid": "Allow use of the key",
  "Effect": "Allow",
  "Principal": {
     "AWS": [
      "arn:aws:iam::12345677890:role/role_name",
     ]
  },
  "Action": [
     "kms:Decrypt",
     "kms:GenerateDataKey",
     "kms:DescribeKey"
  ],
  "Resource": "*"
}

If both source and destination bucket objects are KMS encrypted:

I this case follow both the above steps. Basically, the IAM user/role should be able to do the above actions on KMS of both accounts.

Hope this post helped you in some way. Let me know if this worked for you or not by follow CodeByte on Instagram and to get more quick bits.