CloudFormation Example: Set CodePipeline to trigger with S3 file change events (CloudTrail+CloudWatch)

Earlier this year, AWS announced new, better way for triggering AWS CodePipelines when using S3 as source — Instead of the old-school polling method, you could use CloudWatch Events to trigger the CodePipeline automatically every time source file changes in S3. For anyone wanting to migrate to this new method, they provided helpful migration guide.

Helpful, he said sarcastically, as for anyone using CloudFormation the guide simply states:

You must create:
- The AWS CloudTrail trail.
- The Amazon CloudWatch Events rule.

And this is the extent of it, no examples, no nothing. On top of this, CloudFormation examples for both CloudTrail and CloudWatch Events are pretty much non-existent, let alone for this specific case. And since we are moving to custom source actions (to display BitBucket commit messages in the CodePipeline) we lost the ability to default back to S3 polling, so status quo really wasn’t an option.

Setting this up on CloudFormation was surprisingly annoying (or just personal bad case of Mondays), so I decided to write it down should anyone else find themselves staring at the same situation.

Example

Short explanation — CloudWatch has basic S3 events out of the box, however if you need object specific events CloudTrail is required. In addition, you need to set CloudWatch to look for API event (S3.PutObject) for that specific object and triggering the CodePipeline according.

Note! You need to have CloudTrail S3 bucket set up first.

Parameters:
  S3BucketCodepiPelineSource:
    Description: "The name of the S3 bucket that contains the source artifact. (Must be same region)"
    Type: String
  S3ObjectKeyCodePipelineSource:
    Description: "The filename & path name of the source artifact, such as 'myfolder/myartifact.zip'"
    Type: String
  S3BucketCloudTrail:
    Description: "CloudTrail bucket. (Must be same region)"
    Type: String
Resources:
  # CloudTrail
  # ------------------------------------------------------------
  CloudTrailTrail:
    Type: AWS::CloudTrail::Trail
    Properties:
      IsLogging: true
      S3BucketName: !Ref S3BucketCloudTrail
      EventSelectors:
        - ReadWriteType: WriteOnly
          IncludeManagementEvents: false
          DataResources:
            - Type: AWS::S3::Object
              Values:
                - !Sub "arn:aws:s3:::${S3BucketCodepiPelineSource}/${S3ObjectKeyCodePipelineSource}"
  # CodePipeline
  # ------------------------------------------------------------
  CodePipelinePipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      # ...
  # Events
  # ------------------------------------------------------------
  EventsRule:
    Type: AWS::Events::Rule
    Properties:
      Description: "S3.PutObject for CodePipeline source file"
      State: ENABLED
      EventPattern:
        source:
          - "aws.s3"
        detail:
          eventSource:
            - "s3.amazonaws.com"
          eventName:
            - "PutObject"
          requestParameters:
            bucketName:
              - !Ref S3BucketCodepiPelineSource
            key:
              - !Ref S3ObjectKeyCodePipelineSource
      Targets:
        -
          Id: "TargetCodePipeline"
          Arn: !Sub "arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${CodePipelinePipeline}"
          RoleArn: !GetAtt IAMRoleEventsRule.Arn
  # IAM
  # ------------------------------------------------------------
  IAMRoleEventsRule:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Action: ["sts:AssumeRole"]
            Principal:
              Service: [events.amazonaws.com]
        Version: "2012-10-17"
      Path: /
      Policies:
        - PolicyName: "CodePipelineExecution"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - codepipeline:StartPipelineExecution
                Resource: !Sub "arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${CodePipelinePipeline}"

So there you go, you should now have CodePipeline being triggered through CloudWatch Events instead of the polling for file changes (like it’s 1999). Hope this helps. Cheers.

— By Mikko Tikkanen, Technology Lead at Aller Media Finland