AWS S3 cross-account bucket replication from a bucket with full objects control

Bucket replication is a trivial task.

Cross account bucket replication is a bit more complex but still has good documentation within AWS.

Bucket replication for a bucket that has full control over uploaded objects could be tricky and requires a change of ownership for replicated objects.

The tricky bit

I have worked on a project when data from account1 bucket needed to be replicated in account2 bucket.

The “source bucket” owner has full control and ownership of all objects uploaded to the bucket.

Owner gets FULL_CONTROL. No one else has access rights (default).

{
    "Effect": "Deny",
    "Principal": "*",
    "Action": "s3:PutObject",
    "Resource": "arn:aws:s3:::__SOURCE_BUCKET__/*",
    "Condition": {
        "StringNotEquals": {
            "s3:x-amz-acl": "private"
        }
    }
}

Creating a simple cross-account bucket replication on a source bucket seams to work at the beginning — replication status shown as “COMPLETED”.

The source bucket shows Replication as completed

The object from “Source Account” gets replicated to “Destination Account”; however, the owner of the object is still “Source Account”.
As a result, when you try to access the object in “Destination Account” it gives an “Access Denied” exception and no replica status available.

The destination bucket shows “Access denied” exception for replicated object

This happens because, by default, “source bucket” still owns a replica object located in “destination bucket”

Replication rule without ownership change.

The Solution

To create a working replication between 2 buckets when the source bucket has full object control you need to specify new ownership for the replicated object as a part of the replication rule.

Replication rule with the ownership change.

02 Oct 2020: AWS announced changes to S3 bucket configuration to automatically assume ownership of objects uploaded to their buckets; however, this doesn't include replication.

S3 Object Ownership does not change the behavior of Amazon S3 replication. In replication, the owner of the source object also owns the replica by default.

The “destination account” should be an owner of a replica object in the “destination bucket” to prevent “Access denied”.

Below are the steps overview and a script to make it work.

Steps:

1. In “Source Account” create a role that would be used for the Replication Rule.
   Make sure to include ObjectOwnerOverrideToBucketOwner permission against “Destination bucket”.
   We’re also allowing it on the “Destination bucket” in “Destination account” (as shown in Step 2).
   Without it, the Replication rule would not be able to change the owner for the replica objects.

2. In “Destination account” update “Destination bucket” with a policy that allows the IAM role created in “Source account” to replicate objects in the “destination bucket”.

Make sure to use arn:aws:iam::__SOURCE_ACC_ID__:root and not IAM role for ObjectOwnerOverrideToBucketOwner permission

3. Create a replication rule with “Source bucket” in “source account”.
We’re going to use IAM Role created in the “source account” earlier.

aws s3api put-bucket-replication \
  --replication-configuration file://${ENV}/replication.json \
  --bucket ${SOURCE_BUCKET} \
  --profile ${SOURCE_AWS_PROFILE}

It is important to set

• Account to “Destination account Id”
• Owner to "Destination"

Script

To prevent ClickOps and make it a repeatable process I have created a script with required policy templates.

DmitryNefedov/aws-s3-replication

Pre-requisites:

• aws cli is installed and configured
• aws credentials contain profiles for both “source” and “destination” accounts
• profiles have necessary permissions for buckets access, IAM role creation, put bucket policy, create replication rule etc

Usage

To use the script - clone the project, inspect it and run the script with a number of parameters.

./createS3Replication.sh source_account_profile destination_account_profile env source_bucket_name destination_bucket_name

source_account_profile aws credentials profile for the source account
destination_account_profile aws credentials profile for the destination account
env dev/test/uat etc. Used in the role name
source_bucket_name name of the source bucket in the source account
destination_bucket_name name of destination-bucket in the destination account

e.g.

./createS3Replication.sh main-account replica-account dev important-reports-bucket replica-for-important-reports-bucker

The result

After running the script we have a working replication established between two buckets.

The “source bucket” shows Replication Status “Completed”.

The source bucket shows “COMPLETED” replication status

This time the “destination bucket” has proper Replica Status as well.

The destination bucket has proper access and replication status.

Now we do have a secure working solution to replicate data between buckets in two AWS accounts.

If you have a similar task — give the script a try and let me know how it worked for you.