How to deploy Jovo project with AWS CloudFormation and CI/CD

CloudFormation Stack

Introduction

The document will explain how to create a CloudFormation stack to deploy a Jovo project with CI/CD.

Jovo is the first open-source framework that lets you build voice apps for Amazon Alexa and Google Assistant with one codebase. Including powerful integrations that help you build professional apps faster.
Before deploying a Jovo project with AWS CloudFormation, you can read the course writen by Jan König: How to build a jovo project in 6 step.

Project repository: https://github.com/kouz75/cloudformation-jovo

Part 1 : Resources create by CloudFormation :

• Repository — A Git repository in AWS CodeCommit. When you push a change, the pipeline copies the source code into an Amazon S3 bucket and passes it to the build project.
• Build project — An AWS CodeBuild build that gets the source code from the pipeline and packages the application. The source includes a build specification with commands that install dependencies and prepare an AWS Serverless Application Model (AWS SAM) template for deployment.
• Deployment configuration — The pipeline’s deployment stage defines a set of actions that take the AWS SAM template from the build output, create a change set in AWS CloudFormation, and execute the change set to update the application’s AWS CloudFormation stack.
• AWS CloudFormation stack — The deployment stage uses a template to create a stack in AWS CloudFormation. The template is a YAML-formatted document that defines the resources of the Lambda application. The application includes a Lambda function, a API Gateway and a DynamoDB Table
• Roles — The pipeline, build, and deployment each have a service role that allows them to manage AWS resources. The console creates the pipeline and build roles when you create those resources. You create the role that allows AWS CloudFormation to manage the application stack.

The full AWS CloudFormation template is available on github

1. Repository and Events Rule

CloudFormation will create a repository which will contains Lambda Code, build spec and SAM template.

CodeCommitRepo:
  Description: Creating AWS CodeCommit repository for application source code
  Properties:
    RepositoryDescription: !Join
      - ''
      - - !Ref 'ProjectId'
        - ' project repository'
    RepositoryName: !Ref 'RepositoryName'
  Type: AWS::CodeCommit::Repository

A event rule is use to launch the PipeLine each time an update is done to the master branch.

SourceEvent:
  Properties:
    Description: Rule for Amazon CloudWatch Events to detect changes to the source repository and trigger pipeline execution
    EventPattern:
      detail:
        event:
          - referenceCreated
          - referenceUpdated
        referenceName:
          - master
        referenceType:
          - branch
      detail-type:
        - CodeCommit Repository State Change
      resources:
        - !GetAtt 'CodeCommitRepo.Arn'
      source:
        - aws.codecommit
    Name: !Join
      - '-'
      - - !Ref 'ProjectId'
        - SourceEvent
    State: ENABLED
    Targets:
      - Arn: !Join [':', ['arn:aws:codepipeline',!Ref 'AWS::Region', !Ref 'AWS::AccountId', !Join ['-', [!Ref 'ProjectId','Pipeline']] ]]
        Id: ProjectPipelineTarget
        RoleArn: !GetAtt 'SourceEventRole.Arn'
  Type: AWS::Events::Rule

2. CodeBuild and Artifact Bucket

CodeBuild is a managed service that compiles your source code, runs tests, produces deployable application artifacts. The artifact will be store in the Artifact S3 Bucket
CodeBuild will use the last Amazon Linux 2 image which allow you to use nodejs 10.x or 12.x

CodeBuildProject:
  DependsOn:
    - CodeBuildPolicy
  Properties:
    Artifacts:
      Packaging: zip
      Type: codepipeline
    Description: CodeBuild project
    Environment:
      ComputeType: small
      EnvironmentVariables:
        - Name: S3_BUCKET
          Value: !Ref 'ArtifactS3Bucket'
      Image: aws/codebuild/amazonlinux2-x86_64-standard:2.0
      Type: container
    Name: !Ref 'ProjectId'
    ServiceRole: !Ref 'CodeBuildRole'
    Source:
      Type: codepipeline
  Type: AWS::CodeBuild::Project
ArtifactS3Bucket:
  Type: AWS::S3::Bucket
  Description: Creating Amazon S3 bucket for AWS CodePipeline artifacts
  Properties:
    BucketName: !Join
      - '-'
      - - !Ref 'AWS::Region'
        - !Ref 'AWS::AccountId'
        - !Ref 'ProjectId'
        - pipe
    Tags:
      - Key: APP
        Value: !Ref 'ProjectId'
    VersioningConfiguration:
      Status: Enabled
    PublicAccessBlockConfiguration:
      BlockPublicAcls: true
      BlockPublicPolicy: true
      IgnorePublicAcls: true
      RestrictPublicBuckets: true

3. CodePipeline

AWS CodePipeline create a continuous delivery pipeline for the Lambda function. CodePipeline combines source control, build, and deployment resources to create a pipeline that runs whenever you make a change to your application’s source code.
The AWS SAM template file is a YAML or JSON configuration file that adheres to the open source AWS Serverless Application Model specification. You use the template to declare all of the AWS resources that comprise your serverless application

Deployment PipeLine

Part 2: Roles use by the Stack:

1. LambdaTrustRole:

The role will be use by the Lambda function, it allow the function to :

• Use DynamoDB table.
• write CloudWatch log
• assume lambda role

LambdaTrustRole:
  Type: AWS::IAM::Role
  Description: Creating service role in IAM for AWS Lambda. Lambda function need to read/write to cloudwatch and to use DynamoDB.
  Properties:
    AssumeRolePolicyDocument:
      Statement:
        - Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
              - lambda.amazonaws.com
    Path: /
    Policies:
      - PolicyDocument:
          Statement:
            - Action:
                - logs:CreateLogGroup
                - logs:CreateLogStream
                - logs:PutLogEvents
              Effect: Allow
              Resource: '*'
            - Action:
                - dynamodb:GetItem
                - dynamodb:DeleteItem
                - dynamodb:PutItem
                - dynamodb:UpdateItem
              Effect: Allow
              Resource:
                - !Join
                  - ''
                  - - 'arn:aws:dynamodb:*:*:table/'
                    - !Join ['-', [!Ref 'ProjectId',"session"]]
          Version: 2012-10-17
        PolicyName: LambdaWorkerPolicy
    RoleName: !Join ['-', ['Role',!Ref 'ProjectId', 'Lambda']]

2. CloudFormationTrustRole
The role will be use by CodePipeline, it give CodePipeline access to :

• Upload Artifact to S3 bucket
• Create and manage Lambda function
• Create and manage DynamoDB table
• Create API Gateway

CloudFormationTrustRole:
  Type: AWS::IAM::Role
  Description: Creating service role in IAM for AWS CloudFormation
  Properties:
    AssumeRolePolicyDocument:
      Statement:
        - Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
              - cloudformation.amazonaws.com
    Path: /
    Policies:
      - PolicyDocument:
          Statement:
            - Action:
                - lambda:*
                - apigateway:*
              Effect: Allow
              Resource: '*'
            - Action:
                - dynamodb:CreateTable
                - dynamodb:DeleteTable
                - dynamodb:DescribeTable
                - dynamodb:UpdateTable
                - dynamodb:TagResource
                - dynamodb:UntagResource
              Effect: Allow
              Resource:
                - !Join
                  - ''
                  - - 'arn:aws:dynamodb:*:*:table/'
                    - !Join ['-', [!Ref 'ProjectId',"session"]]
            - Action:
                - iam:PassRole
              Effect: Allow
              Resource:
                - !GetAtt
                  - LambdaTrustRole
                  - Arn
            - Action:
                - cloudformation:CreateChangeSet
              Effect: Allow
              Resource:
                - !Join [':', ['arn:aws:cloudformation', !Ref 'AWS::Region', 'aws:transform/Serverless-2016-10-31']]
            - Action:
                - s3:PutObject
                - s3:GetObject
                - s3:GetObjectVersion
              Effect: Allow
              Resource:
                - !GetAtt ArtifactS3Bucket.Arn
                - !Join ['', [!GetAtt ArtifactS3Bucket.Arn , '/*']]
        PolicyName: CloudFormationRolePolicy
    RoleName: !Join ['-', ['Role',!Ref 'ProjectId','CloudFormation']]

3. CodeBuildPolicy
The policy will be use by CodeBuild.

• Create/Write CloudWatch log
• Put/Get object on Artifact and Data Bucket
• Pull CodeCommit repository

CodeBuildPolicy:
  Type: AWS::IAM::Policy
  Description: Setting IAM policy for service role for Amazon EC2 instances
  Properties:
    PolicyDocument:
      Statement:
        - Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
          Effect: Allow
          Resource: '*'
        - Action:
            - s3:PutObject
            - s3:GetObject
            - s3:GetObjectVersion
          Effect: Allow
          Resource:
            - !GetAtt ArtifactS3Bucket.Arn
            - !Join ['', [!GetAtt ArtifactS3Bucket.Arn , '/*']]
        - Action:
            - codecommit:GitPull
          Effect: Allow
          Resource:
            - !Join [':', ['arn:aws:codecommit',!Ref 'AWS::Region', !Ref 'AWS::AccountId',!Ref 'RepositoryName']]
        - Action:
            - kms:GenerateDataKey*
            - kms:Encrypt
            - kms:Decrypt
          Effect: Allow
          Resource:
            - !Join [':', ['arn:aws:kms',!Ref 'AWS::Region', !Ref 'AWS::AccountId','alias/aws/s3']]
    PolicyName: CodeBuildPolicy
    Roles:
      - !Ref 'CodeBuildRole'

Part 3: Create the Cloudformation Stack

Open CloudFormation and switch to region support by Alexa.
Create a new stack and fill the form:

AWS CloudFormation Stack name and parameters

After validation, you should wait several minutes for stack to be create.
Once the creation done, you will have to sync the repository create by CloudFormation with github.

Part 4: Lambda deployement configuration

1. template.yml

AWS SAM templates are an extension of AWS CloudFormation templates.
SAM Template is use during build process. It will be use by CodeBuild to generate a change set for CloudFormation. the pipeline will create:

• Lambda function which will be call directly by Alexa and via API Gateway for Google assistant.
  It is highly recommended to enable skill ID verification to protect your function from malicious callers.

LambdaJovoFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Join ['-', [!Ref 'ProjectId',"jovo"]]
      Handler: index.handler
      Runtime: nodejs12.x
      MemorySize: 1024
      Timeout: 10
      AutoPublishAlias: v1
      CodeUri: bundle/
      Role: !Join ['', [ "arn:aws:iam::",!Ref 'AWS::AccountId',":role/Role-",!Ref 'ProjectId',"-Lambda"]]
      Events:
        WebHook:
          Type: Api
          Name: !Join ['-', [!Ref 'ProjectId',"api"]]
          StageName: Prod
          Properties:
            Path: /
            Method: any
        AlexaSkillEvent:
          Type: AlexaSkill
#          Properties:
#            SkillId: 'xxx'
      Environment:
        Variables:
          NODE_ENV: prod
      Tags:
        SITE: !Ref 'ProjectId'

• DynamoDB table to store users data.

SessionTable:
  Type: "AWS::DynamoDB::Table"
  DeletionPolicy: Retain
  Properties:
    TableName: !Join ['-', [!Ref 'ProjectId',"session"]]
    AttributeDefinitions:
      -
        AttributeName: "userId"
        AttributeType: "S"
    KeySchema:
      -
        AttributeName: "userId"
        KeyType: "HASH"
    ProvisionedThroughput:
      ReadCapacityUnits: 5
      WriteCapacityUnits: 5
    Tags:
      - Key: SITE
        Value: !Ref 'ProjectId'

2. buildspec.yml

The buildspec is use by AWS CodeBuild. Several action are define:

• Set the environment to unittest.
  Jovo will use unittest configuration file. Sessions generate by unitest will be store localy instead of dynomoDB.
• set nodejs runtime version to 12.x
• install jovo client and node modules
• run unit test.
• generate a zip file of the project
• Uploading Artifact to an S3 Bucket and CloudFormation stack with the correct path to the S3

version: 0.2
environment_variables:
  plaintext:
    NODE_ENV: "unittest"
phases:
  install:
    runtime-versions:
      nodejs: 12
    commands:
      - npm install -g jovo-cli
      - npm install --save-dev
  pre_build:
    commands:
      - npm test
  build:
    commands:
      - npm run bundle
      - aws cloudformation package --template template.yml --s3-bucket $S3_BUCKET --output-template template-export.yml

artifacts:
  type: zip
  files:
    - template-export.yml

Part 5: Install guide

• Create a CloudFormation stack with the template cloudformation.yml the stack must be create in any region support by alexa.
• Sync this repository with your CodeCommit repository create by CloudFormation
• Push your change to your repository. The PipeLine should generate a new version of your lambda function and update CloudFront with the correct Lambda Version.
• Update DialogFlow endpoint with API Gateway url and AlexaSkill endpoint with the lambda arn.

Test and enjoin…