Use the Serverless Framework to Test AWS Microservices
Learn to make use of AWS serverless services for testing
Functional testing of AWS services is slightly challenging compared to normal REST API, web applications, or databases. Let’s see what the major problems are when dealing with AWS services.
Let’s consider a solution running an application using Lambda functions and orchestrated using the step function, storing metadata in DynamoDB and data files in S3.
When building a test-automation framework, if you use Jenkins, CircleCI, GoCD, or any other CI tool to execute tests, then passing and storing user credentials like aws_access_key_id and aws_secret_access_key on the server isn’t a secure way of accessing AWS services — and it has potential security concerns. So in this case, you’d need to assign your CI/CD server running within AWS (EC2/ECS) and assign IAM Roles to access all other AWS services.
This solution also has a limitation: If the services in the application use VPC endpoints, then it becomes really difficult to manage, as typically CI servers in an enterprise are hosted in different AWS accounts (traditionally, called tooling accounts). Visually, it can be represented like this:
How Serverless Frameworks Help to Mitigate These Problems
This framework considers the following assumptions:
- Tests don’t run longer than 15 minutes (current the Lambda max execution time)
- Tests don’t consume more than 3008 MB memory for execution
In this article, I’ll discuss a few of the components required to build and run tests serverlessly. I’ll be leaving out how to build out Alexa Skill for voice commands for test invocation/test execution reports and how to use an Express server for reporting the dashboard, but I’ll return to these topics in future articles.
Step 1
Write your automated tests using any existing framework. In this example, we are using Mocha Chai to show a sample test. Ideally, these would be automated tests to validate your AWS application.
Run the below commands to first run these tests locally:
mkdir serverless-test-framework && cd serverless-test-framework
npm init -y
npm i mocha chai mochawesome moment-timezone aws-sdk fsmocha,chai: Mocha is a test framework, and Chai is the assertion librarymochawesome: A reporting library to produce beautiful HTML reportsmoment-timezone: Used to generate the current date and time for the AEST timezone easilyaws-sdk: Used to interact with services like S3 and SNS to push report files and notificationsfs: Used to read the file contents from a local directory in order to copy over to S3
Create the test file, and copy the below code into it. This is a sample test case.
touch test.jsAt this stage, these tests can be run independently by running the mocha command. This would generate an output like below. Note, there’s not an HTML report generated just yet.
Let’s start writing the code for the Lambda function. First of all, create a lib file that’ll have helper functions to interact with the necessary AWS services required for the framework.
mkdir lib && cd lib
touch aws-manager.jsCopy the below code to the aws-manager.js file. Line 3 has the region value. Please ensure to update it based on your location.
The final step is to create the index.js file required for the Lambda function’s invocation.
// come out of lib directory
cd ..
touch index.jsCopy the below code into index.js. In this file, we’re going to invoke Mocha tests using a programmatic approach.
Note: There are references to S3 and SNS topics that need to be updated at line 40 and 55 once we create our resources in AWS.
This is how your folder should look like at the end of this step. If not, check which step has been missed. (node_modules will be created automatically after the npm i command.)
Step 2
Let’s start building the AWS resources. I assume you already have an account with AWS — if not, it’s free to sign up. Go to the AWS Console, and sign in using your credentials.
Create the S3 bucket
This is required to store the execution reports generated using mochawesome.
Click on “Services,” and click “S3.”
Click the “Create bucket” button, and enter your bucket name. Select the region based on your location, and click the “Create bucket” button on the bottom.
Don’t forget to update name on line 40 in the index.js file.
Create SNS topic
This is used to send notifications to all subscribers if there is a test failure.
Click “Services,” and click “Simple Notification Service” under the Application Integration section. Or search for “SNS” in the search bar.
Click on “Topics” in the left pane, and click the “Create topic” button
Enter the name of the topic, and click on the “Create topic” button on the bottom.
Don’t forget to update name on line 55 in the index.js file.
Let’s add subscribers to this topic. Click on the topic name created above, and click “Create subscription.” This is a very important step to ensure you’ll be receiving notifications upon test failure.
Select “Email” as the protocol, and enter your valid email. Click on the “Create subscription” button on the bottom of the screen.
This will trigger an email to your ID with the subject “AWS Notification — Subscription Confirmation.” Click on the “Confirm subscription” link.
Optional: Create another subscription for your mobile number.
Create a Lambda function
This is used to deploy tests for automated execution. Click “Services,” and click “Lambda” under the Compute section. Or search for “Lambda” in the search bar.
Click the “Create function” button, and you should see following screen:
Enter the function name, and select “Node.js 12.x” (it’s the default selection) for runtime.
The next step is probably the most important section in this entire tutorial. Create an IAM role, which will grant this Lambda function access to the S3 bucket and SNS topic created above.
Expand the Choose section, or create an execution role.
Enter the role name and from the Policy templates drop-down, and select the “Amazon SNS publish policy.”
Click “Create function.”
This is how your Lambda function will look like at first. We’ll deploy tests to this Lambda in Step 3.
Scroll down on the Lambda function page, and under Basic settings, click “Edit.” Increase Timeout to 30 seconds, and click “Save.”
On the Lambda function page, under the execution role, click on “View the <rolename> role.” This will take you to IAM Management Console role page.
Click on the “Attach Policy” button. On the “Attach Permissions” page, search “s3” in the search bar, and select “AmazonS3FullAccess.” Click on “Attach policy” on the bottom of the screen.
Create CloudWatch Event
This will be required to trigger the Lambda function on a scheduled basis.
Click “Services,” and click “CloudWatch” under the Management and Governance section. Or search “CloudWatch” in the search bar.
Click on “Rule”s in the left pane, and click the “Create rule” button.
Click on the “Schedule” radio button, and select “Cron Expression.” Enter 30 13 * * ? * in the textbox. This means this event will occur every day at 13:30 GMT time. You need to convert your local time into GMT to correctly schedule the event.
Click on “Add target” on the right side under the Targets section, and select your Lambda function from the drop-down.
Click on the “Configure details” button on the bottom.
Enter the name and description, and click on the “Create rule” button on the right side.
Note: The state is by default enabled. Don’t disable it.
Step 3
All the AWS resources are created, and the code is ready to be deployed to the Lambda function.
Open the terminal, and cd serverless-test-framework. This is the same directory we created in Step 1. Package up your test code.
zip -qr tests.zip .This would generate a new Zip file with the name tests.zip.
Go back to the AWS Console, and go to the Lambda function created as part of Step 2.
Click on the “Code entry type” drop-down, and select “Upload a .zip file.”
Click on the Upload button, and select the tests.zip file. Click on the “Save” button on the top of the screen.
Important: Click on the “Save” button on the top of the screen intentionally repeated).
Now it’s time to test if the deployment is successful.
Click on the drop- down next to “Test” on the top of the screen and click “Configure test events.”
Enter the name of the event, and leave everything as is. Click the “Create” button.
Click on the “Test” button, and you should see the below green output. This means the Lambda function has been run successfully, and all tests have been executed.
Let’s check the execution report in S3. Open a new tab, and go to the S3 bucket created in Step 2.
In this S3, there should be new folders created automatically, reports > html — contains all HTML reports reports > json — contains corresponding JSON report per execution
Let’s now check if we get a notification when a test fails. Go back to your editor, and let’s intentionally fail one of the tests.
Open test.js, and change line 6 to expect(i).to.not.equal(9).
Execute the Zip command zip -qr tests.zip, and it’ll update the existing tests.zip file.
Go back to the AWS Console Lambda function. Click on “Upload,” and select tests.zip. Click “Save” in the top-right corner.
Click “Test” once it’s saved successfully.
Logs show one failure and nine passes. Also, you can check detailed logs in CloudWatch.
Click on “Monitoring,” and click “View logs in CloudWatch.”
Select the latest stream if you’ve executed multiple times.
Meanwhile, you should have received an email in your inbox and through SMS (depending on your subscriptions).
Step4
One last thing is outstanding: We need to check if the Lambda function executes based on your scheduled trigger in Step 2. Check the logs in CloudWatch, and see if you notice a log stream for that time.
Note: The time mentioned in the Scheduler section is GMT and not your local time.
Conclusion
We’ve covered only a small portion of what’s shown in the Detailed Test Framework Architecture graphic (below). But its a very good start, and you now can have automated tests running in a serverless environment.
You can add more tests to the test.js file, and if you want to run tests against any AWS service, like DynamoDB or RDS, just ensure to update the IAM role and increase the timeout/memory as required.
This is a neat and clean approach to have tests running independently for each application.
And it’s all done! Good job if you managed to create all of the resources and to execute the tests successfully. In my next article, I’ll explain how to build an Alexa skill to execute tests using voice commands. Imagine, saying “Hey Alexa, execute my sample tests” and seeing your test execution start.
Build simple but detailed reporting dashboard for your framework — https://medium.com/javascript-in-plain-english/reporting-dashboard-for-serverless-test-framework-36dd46bf54da
