CodePipeline pipeline for automation Blue/Green deployment with SM Automation runbook

Introduction:

In this post, we have a configuration for Systems Manager automation execution on GitLab webhook event with CodePipeline pipeline.
CodePipeline pipeline consists of stages: Source, Copy-to-s3, Invoke-lambda.
Source: retrieves code changes when a webhook event is sent from GitLab;
Copy-to-s3: files from the Source stage are unzipped before saving and save files to the S3 bucket;
Invoke-lambda: invoke a Lambda function to perform execution of the SM Automation runbook.
AWS CodeStar Gitlab Connections service is not available in all regions, more information here .
Monitoring of the pipeline is done with the notification rule which sends error notifications to the SNS topic.

This post is the second part of my series of posts about Blue/Green deployment on AWS EC2 instances with the Systems Manager Automation runbook, the first part is here, and the third part is here.

About the project:

All infrastructure is created with CloudFormation template infrastructure/codepipeline_pipeline.yaml and has independent deployment from the ec2-bluegreen-deployment stack.

codepipeline_pipeline.yaml template:

AWSTemplateFormatVersion: '2010-09-09'
Description: 'Codepipeline: take source code, store to S3, execute SSM runbook'

Parameters:
  SsmDocumentName:
    Type: String
    Default: 'Ec2BlueGreenDeployment'
  ConnectionName:
    Type: String
    Default: 'gitlab-to-s3'
  S3BucketName:
    Type: String
    Default: ''
  BranchName:
    Type: String
    Default: ''
  FullRepositoryId:
    Type: String
    Default: ''
  CodePipelineName:
    Type: String
    Default: 'execute-blue-green-runbook'
  TopicName:
    Type: String
    Default: 'blue-green-deployment-status'

Resources:
  GitLabConnection:
    Type: 'AWS::CodeStarConnections::Connection'
    Properties:
      ConnectionName: !Ref ConnectionName
      ProviderType: 'GitLab'

  ExecuteSsmRunbookRole:
    Type: 'AWS::IAM::Role'
    Properties:
      RoleName: ExecuteSsmRunbookRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
      Policies:
        - PolicyName: ExecuteSsmRunbook
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - ssm:StartAutomationExecution
                Resource:
                  - !Sub 'arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:*'
        - PolicyName: ReturnMessageToCodepipeline
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - codepipeline:PutJobFailureResult
                  - codepipeline:PutJobSuccessResult
                Resource: '*'


  ExecuteSsmRunbook:
    Type: 'AWS::Lambda::Function'
    Properties:
      FunctionName: ExecuteSsmRunbook
      Handler: index.lambda_handler
      Role: !GetAtt ExecuteSsmRunbookRole.Arn
      Environment:
        Variables:
          SsmDocumentName: !Ref SsmDocumentName
      Runtime: python3.12
      Timeout: 10
      Code:
        ZipFile: |
          import boto3
          import os
          from botocore.exceptions import ClientError

          def lambda_handler(event, context):
              try:
                  aws_region = os.environ.get('AWS_REGION')
                  SsmDocumentName = os.environ['SsmDocumentName']
                  codepipeline = boto3.client('codepipeline', region_name=aws_region)
                  ssm = boto3.client('ssm')
                  
                  # Retrieve the Job ID from the Lambda action
                  job_id = event.get('CodePipeline.job', {}).get('id')

                  # Notify CodePipeline of a successful job
                  def put_job_success(message):
                      params = {
                          'jobId': job_id,
                          'executionDetails': {
                              'summary': str(message),
                              'percentComplete': 100,
                              'externalExecutionId': context.aws_request_id
                          }
                      }
                      codepipeline.put_job_success_result(**params)
                      print(message)

                  # Notify CodePipeline of a failed job
                  def put_job_failure(message):
                      params = {
                          'jobId': job_id,
                          'failureDetails': {
                              'message': str(message),
                              'type': 'JobFailed',
                              'externalExecutionId': context.aws_request_id
                          }
                      }
                      codepipeline.put_job_failure_result(**params)
                      print(message)

                  # Start the SSM Automation runbook execution
                  response = ssm.start_automation_execution(DocumentName=SsmDocumentName)
                  execution_id = response['AutomationExecutionId']

                  put_job_success(f'Started SSM Automation execution with ID: {execution_id}')

              except Exception as e:
                  # Log any unexpected errors and fail the job
                  error_message = f'Error: {str(e)}'
                  print(error_message)
                  put_job_failure(error_message)
                  raise e

  SSMAutomationTriggerPermissions:
    Type: AWS::Lambda::Permission
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName: !GetAtt ExecuteSsmRunbook.Arn
      Principal: 'codepipeline.amazonaws.com'

  CodePipelineRole:
    Type: 'AWS::IAM::Role'
    Properties:
      RoleName: 'CodePipelineRole'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: 'Allow'
            Principal:
              Service: 'codepipeline.amazonaws.com'
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: 'CodePipelineFullAccess'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Allow'
                Action:
                  - 'codepipeline:*'
                Resource: !Sub 'arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:*'
        - PolicyName: 'S3FullAccess'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Allow'
                Action:
                  - s3:PutObject
                  - s3:GetObject
                  - s3:ListBucket
                Resource: !Sub 'arn:${AWS::Partition}:s3:::*'
        - PolicyName: 'LambdaInvokeAccess'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Allow'
                Action:
                  - lambda:InvokeFunction
                Resource: !Sub 'arn:${AWS::Partition}:lambda:${AWS::Region}:${AWS::AccountId}:function:*'
        - PolicyName: 'CodeStarSourceConnectionAccess'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Allow'
                Action:
                  - codestar-connections:UseConnection
                Resource: !Sub 'arn:${AWS::Partition}:codestar-connections:${AWS::Region}:${AWS::AccountId}:connection/*'
        - PolicyName: 'SendNotificationsBySns'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: 'Allow'
                Action:
                  - sns:Publish
                Resource:  '*'

  CodePipelineTriggerRunbook:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      Name: !Ref CodePipelineName
      RoleArn: !GetAtt CodePipelineRole.Arn
      ArtifactStore:
        Type: S3
        Location: !Ref S3BucketName
      Stages:
        - Name: Source
          Actions:
            - Name: Source
              ActionTypeId:
                Category: Source
                Owner: AWS
                Provider: CodeStarSourceConnection
                Version: '1'
              RunOrder: 1
              Configuration:
                BranchName: !Ref BranchName
                ConnectionArn: !Ref 'GitLabConnection'
                DetectChanges: 'true'
                FullRepositoryId: !Ref FullRepositoryId
                OutputArtifactFormat: CODE_ZIP
              OutputArtifacts:
                - Name: SourceArtifact
              Namespace: SourceVariables
        - Name: Copy-to-s3
          Actions:
            - Name: copy-to-s3
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Provider: S3
                Version: '1'
              RunOrder: 1
              Configuration:
                BucketName: !Ref S3BucketName
                Extract: 'true'
              InputArtifacts:
                - Name: SourceArtifact
        - Name: Invoke-lambda
          Actions:
            - Name: invoke-lambda
              ActionTypeId:
                Category: Invoke
                Owner: AWS
                Provider: Lambda
                Version: '1'
              RunOrder: 1
              Configuration:
                FunctionName: !Ref ExecuteSsmRunbook
              OutputArtifacts:
                - Name: Message
              InputArtifacts:
                - Name: SourceArtifact
      PipelineType: 'V2'

  CodePipelineNotificationRule:
    Type: 'AWS::CodeStarNotifications::NotificationRule'
    Properties:
      DetailType: 'BASIC'
      EventTypeIds:
        - 'codepipeline-pipeline-pipeline-execution-failed'
        - 'codepipeline-pipeline-stage-execution-failed'
        - 'codepipeline-pipeline-action-execution-failed'
      Name: SsmPipelineNotificationRule
      Resource: !Sub 'arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:${CodePipelineName}'
      Targets:
        - TargetAddress: !Sub 'arn:${AWS::Partition}:sns:${AWS::Region}:${AWS::AccountId}:${TopicName}'
          TargetType: 'SNS'

Deployment and infrastructure schemas:

CodePipeline pipeline stages

Infrastructure schema

Prerequisites:

Before you start, make sure the following requirements are met:
- An AWS account with permissions to create resources.
- AWS CLI installed on your local machine.

Deployment:

1. Clone the repository (if you don’t have already cloned it), navigate to the cloned repository, and push the repository to your remote repository.

git clone https://gitlab.com/Andr1500/ssm_runbook_bluegreen.git

2. Fill all necessary Parameters in the infrastructure/codepipeline_pipeline.yaml file, go to the infrastructure directory, and create CloudFormation stack.

aws cloudformation create-stack \
    --stack-name codepipeline-pipeline\
    --template-body file://codepipeline_pipeline.yaml \
    --capabilities CAPABILITY_NAMED_IAM --disable-rollback

3. Update CodeStar pending connection. Open the AWS Console, next go: CodePipeline -> Settings -> Connections -> choose the created connection in pending status -> Update pending connection -> depends on your provider make authorisation, and grant necessary access to the repository. More information about CodeStar connections here .

4. For deployment of a new version of the application — make changes in application/index.html, commit changes, and push it to your remote repository. The deployment process will be started automatically and in case of any issues with the deployment, both the CodePipeline pipeline and runbook, you will receive an email with issues details.

Conclusion:

In this post, we showed how the CodePipeline pipeline can automate Blue/Green deployment with Application Load Balancer’s weighted target group feature realised with the Systems Manager Automation runbook.

If you found this post helpful and interesting, please click the clap button below to show your support.