In one previous post we wrote Lambda function to put customer info to DynamoDB via REST API. These are all done from AWS console.
Instead of using console, many would be using CloudFormation template file. But CloudFormation template file has a relative deep learning curve for beginners. Also it can grow huge with all the detail resource attributes in which case it can be very difficult to maintain.
Good news, if you have just started with CloudFormation or suffered with the draw backs you can not jump to AWS CDK v2. CDK will generate CloudFormation template file from your Python or other programming language code. AWS can further use the files to maintain your infrastructure.
AWS CDK 2.0 has released in Dec 2021 with significant improvement to make it easy to use. For example, defining a role for Lambda to read DynamoDB table can use 30+ lines in template file while it can be only one line code in CDK.
Checkout following details for how to get started!
1 Install AWS CLI and Node.js/npm
Use this doc link to download and install awscliv2. After that, you need configure local AWS env with your access id and secret id.
[fengli@mycentos8 aws]curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
[fengli@mycentos8 aws]unzip awscliv2.zip
[fengli@mycentos8 aws]sudo ./aws/install[fengli@mycentos8 aws] aws --version
aws-cli/2.7.19 Python/3.9.11 Linux/4.18.0-408.el8.x86_64 exe/x86_64.centos.8 prompt/off[fengli@mycentos8 aws]$ aws configure
AWS Access Key ID [****************XZEF]: xxx
AWS Secret Access Key [****************qAiI]: yyy
Default region name [us-east-1]:
Default output format [json]:
Next, install latest Node.js which has built-in npm.
[root@mycentos8 ~]# dnf install nodejs
...
[root@mycentos8 ~]# node --version
v16.17.0
[root@mycentos8 ~]# npm --version
8.15.0Assuming that you have Python and git installed already.
2 Install AWS CDK
[root@mycentos8 ~]# npm install aws-cdk-libadded 2 packages, and audited 22 packages in 19sfound 0 vulnerabilities
[root@mycentos8 ~]# cdk --version
2.35.0 (build 5c23578)
3 Initialize a CDK application
[fengli@mycentos8 newapp]$ pwd
/home/fengli/workspace/cdk-work/newapp#### there are 3 application templates you can start with
[fengli@mycentos8 newapp]$ cdk init
Available templates:
* app: Template for a CDK Application
└─ cdk init app --language=[csharp|fsharp|go|java|javascript|python|typescript]
* lib: Template for a CDK Construct Library
└─ cdk init lib --language=typescript
* sample-app: Example CDK Application with some constructs
└─ cdk init sample-app --language=[csharp|fsharp|go|java|javascript|python|typescript][fengli@mycentos8 newapp]$ cdk init sample-app --language=python
Applying project template app for python
...[fengli@mycentos8 newapp]$ ls -la
total 32
drwxrwxr-x. 6 fengli fengli 4096 Aug 25 19:42 .
drwxrwxr-x. 8 fengli fengli 100 Aug 25 19:42 ..
-rw-rw-r--. 1 fengli fengli 149 Aug 25 19:42 app.py
-rw-rw-r--. 1 fengli fengli 1246 Aug 25 19:42 cdk.json
drwxrwxr-x. 8 fengli fengli 166 Aug 25 19:42 .git
-rw-rw-r--. 1 fengli fengli 239 Aug 25 19:42 .gitignore
drwxrwxr-x. 2 fengli fengli 48 Aug 25 19:42 newapp
-rw-rw-r--. 1 fengli fengli 1972 Aug 25 19:42 README.md
-rw-r--r--. 1 fengli fengli 14 Aug 25 19:42 requirements-dev.txt
-rw-r--r--. 1 fengli fengli 47 Aug 25 19:42 requirements.txt
-rw-r--r--. 1 fengli fengli 437 Aug 25 19:42 source.bat
drwxrwxr-x. 3 fengli fengli 37 Aug 25 19:42 tests
drwxrwxr-x. 5 fengli fengli 74 Aug 25 19:42 .venv
So what the initialization does are as followings:
(1) file app.py and newapp directory
app.py is entry file of this CDK application. It looks for your stack code and create CDK application at local. This CDK application manages CDK stack/s at local and deploy them to AWS.
Note, there is no CDK application concept on AWS side. there is a 1–1 mapping for CDK stack and AWS CloudFormation stack.
[fengli@mycentos8 newapp]$ cat app.py
#!/usr/bin/env python3import aws_cdk as cdkfrom newapp.newapp_stack import NewappStackapp = cdk.App()
NewappStack(app, "newapp")app.synth()
CDK initialization creates a subdirectory to hold stack code and it names the subdirectory after the current directory. In my case, I have directory “/home/fengli/workspace/cdk-work/newapp/newapp”. A little confusing though — could it be better to name it like “stack”?
So directory “/home/fengli/workspace/cdk-work/newapp/newapp” contains a Python file newapp_stack.py in which you write code to define CDK stack/s.
[fengli@mycentos8 newapp]$ pwd
/home/fengli/workspace/cdk-work/newapp/newapp
[fengli@mycentos8 newapp]$ ll
-rw-rw-r--. 1 fengli fengli 597 Aug 25 10:42 newapp_stack.py[fengli@mycentos8 newapp]$ cat newapp_stack.py
from constructs import Construct
from aws_cdk import (
Duration,
Stack,
aws_iam as iam,
aws_sqs as sqs,
aws_sns as sns,
aws_sns_subscriptions as subs,
)class NewappStack(Stack): def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs) queue = sqs.Queue(
self, "NewappQueue",
visibility_timeout=Duration.seconds(300),
) topic = sns.Topic(
self, "NewappTopic"
) topic.add_subscription(subs.SqsSubscription(queue))
CDK stack is defined as a Python class which is inherited from Class Stack in aws_cdk package. All AWS resources are classes in aws_cdk package too. In your CDK stack class, you can define AWS resource like SQS, SNS etc. as shown in above sample code.
To me this is the power of CDK compared to CloudFormation template file: use Python classes and methods to define AWS resources. (again, under the hood, CDK run Python code to generate CloudFormation templates to deploy stack/resources to AWS.)
(2) requirements.txt
CDK uses Python virtual environment to run your Python code at local and then deploys resource stack to AWS. So CDK libraries need to be installed in the environment first.
Two libraries are needed: aws-cdk-lib and constructs. They can be installed using this requirements.txt file. (It’ll be great if CDK does this installation automatically when it creates the virtual env during “cdk init…”)
(.venv) [fengli@mycentos8 my-app]$ cat requirements.txt
aws-cdk-lib==2.35.0
constructs>=10.0.0,<11.0.0(3) Python virtual env in .venv directory
So next we’ll source the virtual env and install CDK Python libraries.
[fengli@mycentos8 my-app]$ source .venv/bin/activate
(.venv) [fengli@mycentos8 my-app]$ pip install -r requirements.txt
Collecting aws-cdk-lib==2.35.0
...(4) Remember to do git initialization in this directory
4 Modify sample application code with your resources
On top of the initialized “sample-app” project/framework, we’ll try to figure out how to create API Gateway, Lambda, DynamoDB and related role/permissions.
4.1 CDK API reference
CDK 2 documentation can be found at https://docs.aws.amazon.com/cdk/api/v2/.
We’ll be using aws_apigateway, aws_lambda and aws_dynamodb.
4.2 Launch VS Code
[fengli@mycentos8 newapp]$ pwd
/home/fengli/workspace/cdk-work/newapp
[fengli@mycentos8 newapp]$ code .If VS Code tells that modules “aws_cdk” and “constructs” “could not be resolved”, go search “Python: select interpreter” option in Command Palette (Ctrl+Shift+p). Make sure it’s using the virtual env CDK initialized with required libraries installed.
4.3 Coding for needed resources
You can look up CDK API documentation or find sample code from sample code in this GitHub link.
In our case, we’ll only have two changes on top of the sample app. See following code on GitHub link.
(1) newapp_stack.py (modified from initialized sample code)
(2) lambda/customer_lambda.py (new created directory and file)
As we can see, the code is simple: creating class instances with basic properties, calling object “grant_write_data” method to set permission/role for services, that’s it.
If we do a “cdk diff” at this point, CDK tells differences between local resource stack from our code and what’s currently in AWS. We can see beyond what we have in code, there are IAM, Permissions, Metada etc. a lot of other dependencies are ready to be created to AWS.
PS C:\Users\feng\workspace\cdk-work\newapp> cdk diff
Stack newapp
...
Resources
[+] AWS::DynamoDB::Table customer_table_cdk customertablecdkB993F564
[+] AWS::IAM::Role customer_lambda_cdk/ServiceRole customerlambdacdkServiceRole9A9C44DA
[+] AWS::IAM::Policy customer_lambda_cdk/ServiceRole/DefaultPolicy customerlambdacdkServiceRoleDefaultPolicyC7E5A5B0
[+] AWS::Lambda::Function customer_lambda_cdk customerlambdacdk3230A15D
[+] AWS::ApiGateway::RestApi customer_api_cdk customerapicdk328D08FB
[+] AWS::IAM::Role customer_api_cdk/CloudWatchRole customerapicdkCloudWatchRole15559700
[+] AWS::ApiGateway::Account customer_api_cdk/Account customerapicdkAccountA16308A9
[+] AWS::ApiGateway::Deployment customer_api_cdk/Deployment customerapicdkDeploymentF95E105E8430dcb484fde63595cfd16069e57347
[+] AWS::ApiGateway::Stage customer_api_cdk/DeploymentStage.prod customerapicdkDeploymentStageprod9D320359
[+] AWS::ApiGateway::Resource customer_api_cdk/Default/add_customer customerapicdkaddcustomerA8AA64DB
[+] AWS::Lambda::Permission customer_api_cdk/Default/add_customer/POST/ApiPermission.newappcustomerapicdkBB5FF911.POST..add_customer customerapicdkaddcustomerPOSTApiPermissionnewappcustomerapicdkBB5FF911POSTaddcustomer6BBDAFB6
[+] AWS::Lambda::Permission customer_api_cdk/Default/add_customer/POST/ApiPermission.Test.newappcustomerapicdkBB5FF911.POST..add_customer customerapicdkaddcustomerPOSTApiPermissionTestnewappcustomerapicdkBB5FF911POSTaddcustomer85C2E1BB
[+] AWS::ApiGateway::Method customer_api_cdk/Default/add_customer/POST customerapicdkaddcustomerPOSTE23568735 (Optional)Generate CloudFormation template file
Now we explicitly create CloudFormation template file based on our code if you want to look at details. But it’s optional we can go ahead to deploy our resource stack.
PS C:\Users\feng\workspace\cdk-work\newapp> cdk synthSimply run “cdk deploy” command to deploy to AWS.
PS C:\Users\feng\workspace\cdk-work\newapp> cdk deploy
...Outputs:
newapp.customerapicdkEndpoint7Bxxxxxx = https://xxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/
In the meanwhile, we can check what going on in CloudFormation from AWS console: our stack “newapp” is being deployed.
Once it’s finished, we can double check API Gateway, Lambda function and DynamoDB as well as roles/permissions — those are a bundle managed by our CloudFormation stack.
Actually, CDK creates a “Lambda Serverless Application” for us — it’s not only a “Lambda Function” anymore. This “Lambda Serverless Application” includes CDK Metadata, APIGateway API and Account, IAM role, Lambda Function and DynamoDB table. We can manage them as whole.
Note, it seems AWS has changed my DynamoDB table name and Lambda function name: removing “_”(underscore) and adding random letters. That doesn’t impact our functionalities though.
7 Test with Postman
Similarly as previous post, tryout following test to add one customer by calling this API…
API end point can be found on “Lambda serverless application” GUI or “cdk deploy” output. It looks like https://xxxxxx.execute-api.us-east-1.amazonaws.com/prod/add_customer
Our simple payload
{
"customer_id": "1",
"name": "John Doe",
"email": "john.doe@abc.com"
}
It seems it works…check items in DynamoDB table for the record
Note,
1 When using cdk CLI, if you see following warnings and have timeout after that you need to upgrade cdk cli from current 2.35 to 2.38 which works in my cases.
(.venv) [fengli@mycentos8 newapp]$ cdk diff
Stack newapp
current credentials could not be used to assume 'arn:aws:iam::xxxxxx:role/cdk-hnb6xxxx-lookup-role-xxxxxx-us-east-1', but are for the right account. Proceeding anyway.
...<timeout>(.venv) [fengli@mycentos8 newapp]$ sudo npm install -g aws-cdk
2 It’s strange when using CDK CLI on CentOS 8, it failed to upgrade to 2.38 so “cdk diff”, “cdk deploy” in this post have been done in a Windows 10 env using PowerShell.
Happy Reading!
