AWS CloudFormation stack
AWS CloudFormation stack
AWS CloudFormation Stack

Merge audio files with Lambda@Edge/AWS CloudFormation and CI/CD

Stephane Couzinier
Nov 21, 2019 · 9 min read

Introduction

Why do we want to join audio files ?

Part 1 : Resources create by CloudFormation :

CodeCommitRepo:
Description: Creating AWS CodeCommit repository for application source code
Properties:
RepositoryDescription: !Join
- ''
- - !Ref 'ProjectId'
- ' project repository'
RepositoryName: !Ref 'RepositoryName'
Type: AWS::CodeCommit::Repository
SourceEvent:
Properties:
Description: Rule for Amazon CloudWatch Events to detect changes to the source repository and trigger pipeline execution
EventPattern:
detail:
event:
- referenceCreated
- referenceUpdated
referenceName:
- master
referenceType:
- branch
detail-type:
- CodeCommit Repository State Change
resources:
- !GetAtt 'CodeCommitRepo.Arn'
source:
- aws.codecommit
Name: !Join
- '-'
- - !Ref 'ProjectId'
- SourceEvent
State: ENABLED
Targets:
- Arn: !Join [':', ['arn:aws:codepipeline',!Ref 'AWS::Region', !Ref 'AWS::AccountId', !Join ['-', [!Ref 'ProjectId','Pipeline']] ]]
Id: ProjectPipelineTarget
RoleArn: !GetAtt 'SourceEventRole.Arn'
Type: AWS::Events::Rule

2. CodeBuild and Artifact Bucket

CodeBuildProject:
DependsOn:
- CodeBuildPolicy
Properties:
Artifacts:
Packaging: zip
Type: codepipeline
Description: CodeBuild project
Environment:
ComputeType: small
EnvironmentVariables:
- Name: S3_BUCKET
Value: !Ref 'ArtifactS3Bucket'
- Name: AUDIO_FILE_BUCKET
Value: !Join ['-', [!Ref 'AWS::Region',!Ref 'AWS::AccountId',!Ref 'ProjectId','data']]
Image: aws/codebuild/nodejs:10.14.1
Type: container
Name: !Ref 'ProjectId'
ServiceRole: !Ref 'CodeBuildRole'
Source:
Type: codepipeline
Type: AWS::CodeBuild::Project
ArtifactS3Bucket:
Type: AWS::S3::Bucket
Description: Creating Amazon S3 bucket for AWS CodePipeline artifacts
Properties:
BucketName: !Join
- '-'
- - !Ref 'AWS::Region'
- !Ref 'AWS::AccountId'
- !Ref 'ProjectId'
- pipe
Tags:
- Key: APP
Value: !Ref 'ProjectId'
VersioningConfiguration:
Status: Enabled
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
AWS CodePipeline, deploy to AWS Lambda@edge
AWS CodePipeline, deploy to AWS Lambda@edge
Deployment PipeLine
ProjectPipeline:
DependsOn:
- LambdaTrustRole
- CodePipelineTrustRole
- ArtifactS3Bucket
Properties:
ArtifactStore:
Location: !Ref 'ArtifactS3Bucket'
Type: S3
Name: !Join
- '-'
- - !Ref 'ProjectId'
- Pipeline
RoleArn: !GetAtt
- CodePipelineTrustRole
- Arn
Stages:
- Actions:
- ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
BranchName: master
PollForSourceChanges: false
RepositoryName: !Ref 'RepositoryName'
InputArtifacts: [
]
Name: ApplicationSource
OutputArtifacts:
- Name: !Join
- '-'
- - !Ref 'ProjectId'
- SourceArtifact
RunOrder: 1
Name: Source
- Actions:
- ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref 'ProjectId'
InputArtifacts:
- Name: !Join
- '-'
- - !Ref 'ProjectId'
- SourceArtifact
Name: PackageExport
OutputArtifacts:
- Name: !Join
- '-'
- - !Ref 'ProjectId'
- BuildArtifact
RunOrder: 1
Name: Build
- Actions:
- ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: 1
Configuration:
ActionMode: CHANGE_SET_REPLACE
Capabilities: CAPABILITY_IAM
ChangeSetName: pipeline-changeset
ParameterOverrides: !Join
- ''
- - '{"ProjectId":"'
- !Ref 'ProjectId'
- '"}'
RoleArn: !GetAtt
- CloudFormationTrustRole
- Arn
StackName: !Join
- '-'
- - !Ref 'ProjectId'
- lambda
TemplatePath: !Join
- ''
- - !Ref 'ProjectId'
- -BuildArtifact
- ::template-export.yml
InputArtifacts:
- Name: !Join
- '-'
- - !Ref 'ProjectId'
- BuildArtifact
Name: GenerateChangeSet
OutputArtifacts: [
]
RunOrder: 1
- ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: 1
Configuration:
ActionMode: CHANGE_SET_EXECUTE
ChangeSetName: pipeline-changeset
StackName: !Join
- '-'
- - !Ref 'ProjectId'
- lambda
InputArtifacts: [
]
Name: ExecuteChangeSet
OutputArtifacts: [
]
RunOrder: 2
Name: Deploy
Type: AWS::CodePipeline::Pipeline

Part 2: Roles use by the Stack:

LambdaTrustRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- edgelambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
- arn:aws:iam::aws:policy/service-role/AWSConfigRulesExecutionRole
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource: '*'
- Action:
- s3:GetObject
- s3:PutObject
- s3:PutObjectTagging
Effect: Allow
Resource: !Join ['', ['arn:aws:s3:::', !Join ['-', [!Ref 'AWS::Region',!Ref 'AWS::AccountId',!Ref 'ProjectId','data']] , /*]]
Version: 2012-10-17
PolicyName: LambdaWorkerPolicy
RoleName: !Join ['-', ['Role',!Ref 'ProjectId', 'Lambda']]
CloudFormationTrustRole:
Type: AWS::IAM::Role
Description: Creating service role in IAM for AWS CloudFormation
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service:
- cloudformation.amazonaws.com
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
Effect: Allow
Resource:
- !GetAtt ArtifactS3Bucket.Arn
- !Join ['', [!GetAtt ArtifactS3Bucket.Arn , '/*']]
- Action:
- lambda:*
Effect: Allow
Resource: '*'
- Action:
- s3:*
Effect: Allow
Resource:
- !Join ['', ['arn:aws:s3:::', !Join ['-', [!Ref 'AWS::Region',!Ref 'AWS::AccountId',!Ref 'ProjectId','data']] ]]
- Action:
- cloudfront:*
Effect: Allow
Resource: '*'
- Action:
- iam:PassRole
Effect: Allow
Resource:
- !GetAtt
- LambdaTrustRole
- Arn
- Action:
- cloudformation:CreateChangeSet
Effect: Allow
Resource:
- arn:aws:cloudformation:us-east-1:aws:transform/Serverless-2016-10-31
PolicyName: CloudFormationRolePolicy
RoleName: !Join ['-', ['Role',!Ref 'ProjectId','CloudFormation']]
CodeBuildPolicy:
Type: AWS::IAM::Policy
Description: Setting IAM policy for service role for Amazon EC2 instances
Properties:
PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource: '*'
- Action:
- s3:PutObject
- s3:GetObject
- s3:GetObjectVersion
Effect: Allow
Resource:
- !GetAtt ArtifactS3Bucket.Arn
- !Join ['', [!GetAtt ArtifactS3Bucket.Arn , '/*']]
- !Join ['', ['arn:aws:s3:::', !Join ['-', [!Ref 'AWS::Region',!Ref 'AWS::AccountId',!Ref 'ProjectId','data']] ]]
- !Join ['', ['arn:aws:s3:::', !Join ['-', [!Ref 'AWS::Region',!Ref 'AWS::AccountId',!Ref 'ProjectId','data']] , '/*']]
- Action:
- codecommit:GitPull
Effect: Allow
Resource:
- !Join [':', ['arn:aws:codecommit',!Ref 'AWS::Region', !Ref 'AWS::AccountId',!Ref 'RepositoryName']]
- Action:
- kms:GenerateDataKey*
- kms:Encrypt
- kms:Decrypt
Effect: Allow
Resource:
- !Join [':', ['arn:aws:kms',!Ref 'AWS::Region', !Ref 'AWS::AccountId','alias/aws/s3']]
PolicyName: CodeBuildPolicy
Roles:
- !Ref 'CodeBuildRole'
S3BucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref 'AudioFileS3Bucket'
PolicyDocument:
Statement:
- Effect: Allow
Principal:
CanonicalUser:
Fn::GetAtt: [ CFS3OriginAccessIdentity , S3CanonicalUserId ]
Action: "s3:GetObject"
Resource: !Sub "${AudioFileS3Bucket.Arn}/*"

Part 3: Create the Cloudformation Stack

AWS CloudFormation Create stack
AWS CloudFormation Create stack
Create a CloudFormation stack from a template
AWS CloudFormation Stack spécification
AWS CloudFormation Stack spécification
Configure CloudFormation. ProjectId will be use for Lambda/S3/Role Arn
Image for post
Image for post
Before creating the stack, don’t forget to confirm that CF will create IAM resources.

Part 4: Lambda deployement configuration

Resources:
AudioLambdaEdgeFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Ref 'ProjectId'
Handler: index.handler
Runtime: nodejs10.x
MemorySize: 3008
Timeout: 10
AutoPublishAlias: live
Role:
Fn::ImportValue:
!Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', 'LambdaTrustRole']]
Tags:
SITE: !Ref 'ProjectId'
AudioFileS3Bucket:
Type: AWS::S3::Bucket
Description: Creating Amazon S3 bucket to store audio files
Properties:
BucketName: !Join ['-', [ !Ref 'AWS::Region', !Ref 'AWS::AccountId', !Ref 'ProjectId', 'data']]
Tags:
- Key: APP
Value: !Ref 'ProjectId'
VersioningConfiguration:
Status: Enabled
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
S3BucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref 'AudioFileS3Bucket'
PolicyDocument:
Statement:
- Effect: Allow
Principal:
CanonicalUser:
Fn::GetAtt: [ CFS3OriginAccessIdentity , S3CanonicalUserId ]
Action: "s3:GetObject"
Resource: !Sub "${AudioFileS3Bucket.Arn}/*"
CFS3Distribution:
DependsOn: AudioLambdaEdgeFunction
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Origins:
- DomainName: !Join ['', [!Ref 'AudioFileS3Bucket', '.s3.amazonaws.com']]
Id: myS3Origin
S3OriginConfig:
OriginAccessIdentity: !Join ['',['origin-access-identity/cloudfront/', !Ref 'CFS3OriginAccessIdentity'] ]
Enabled: 'true'
Comment: !Ref 'ProjectId'
DefaultRootObject: index.html
DefaultCacheBehavior:
LambdaFunctionAssociations:
- EventType: origin-response
LambdaFunctionARN: !Ref AudioLambdaEdgeFunction.Version
AllowedMethods:
- GET
- HEAD
- OPTIONS
TargetOriginId: myS3Origin
ForwardedValues:
QueryString: 'false'
Cookies:
Forward: none
Headers:
- Origin
- Access-Control-Request-Headers
- Access-Control-Request-Method
ViewerProtocolPolicy: redirect-to-https
HttpVersion: http2
ViewerCertificate:
CloudFrontDefaultCertificate: true
Tags:
- Key: APP
Value: !Ref 'ProjectId'
CFS3OriginAccessIdentity:
Type: "AWS::CloudFront::CloudFrontOriginAccessIdentity"
Properties:
CloudFrontOriginAccessIdentityConfig:
Comment: !Ref 'ProjectId'
version: 0.2
phases:
install:
commands:
#- npm uninstal aws-sdk
#- npm uninstal aws-sdk-mock
- npm install --save-dev


pre_build:
commands:
#generate config file
- echo {\"audioBucket\"':' \"$AUDIO_FILE_BUCKET\"} > ./config.json
- npm test
#removes “extraneous” packages
- npm prune --production
#Delete unused binary to reduce package size.
- rm -rf node_modules/ffmpeg-static/bin/darwin
- rm -rf node_modules/ffmpeg-static/bin/win32
- rm -rf node_modules/ffprobe-static/bin/darwin
- rm -rf node_modules/ffprobe-static/bin/win32
- rm -rf node_modules/ffmpeg-static/bin/linux/ia32
- rm -rf node_modules/ffmpeg-static/bin/linux/arm
- rm -rf node_modules/ffmpeg-static/bin/linux/arm64
- rm -rf node_modules/ffprobe-static/bin/linux/ia32
- rm -rf node_modules/ffprobe-static/bin/linux/arm
build:
commands:
- aws s3 cp --recursive ./assets s3://$AUDIO_FILE_BUCKET/assets/
- aws cloudformation package --template template.yml --s3-bucket $S3_BUCKET --output-template template-export.yml

artifacts:
type: zip
files:
- template-export.yml

Part 5: Install guide

shirkalab

ShirkaLAB is a voice application agency.

Stephane Couzinier

Written by

CTO of ShirkaLAB Agency

shirkalab

shirkalab

ShirkaLAB is a voice application agency. We design, create, develop, host and manage custom Alexa skills for leading brands and publishers. ShirkaLAB helps publishers to track and monetize their skills and actions on Google

Stephane Couzinier

Written by

CTO of ShirkaLAB Agency

shirkalab

shirkalab

ShirkaLAB is a voice application agency. We design, create, develop, host and manage custom Alexa skills for leading brands and publishers. ShirkaLAB helps publishers to track and monetize their skills and actions on Google

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store