Mastering AWS Serverless Architecture: A Step-by-Step Guide with GitHub Actions

Introduction

In this tutorial, I will delve into the fundamentals of AWS Lambda Functions and AWS API Gateway, focusing on enabling automated deployments and continuous integration/continuous deployment (CI/CD) processes. To follow this hands-on guide, you will need:

1. An AWS Account with IAM privileges or root access.
2. A GitHub Account.

I assume that you already have a basic understanding of Lambda Functions and API Gateways. Our objective is to build a backend system that takes a complete Google Address as input and returns the place’s ratings. To achieve this, we’ll utilize the Google Places API. While I won’t delve deeply into the backend system’s code in this tutorial, we’ll focus on the operational concepts.

Use Case

My use case involves creating a Lambda Function that triggers on an API Gateway call. The Lambda Function will interact with the Google Places API. I’ve also introduced a caching mechanism using DynamoDB to reduce the number of requests to the Google Places API, ultimately saving costs.

For reference, I’ll provide a high-level design diagram to the backed infrastructure and a link to the GitHub repository. You can choose to explore both the backend system and DevOps aspects or focus solely on the DevOps side. If you choose the latter, consider the code within the Lambda Function as a black box.

HLD Diagram for the flow of the program

Github URL : Navigate to the Github Link for this project

Your Guide to AWS Serverless Mastery: Essential Steps

Step 1: Creating the Lambda Function and Establishing API Gateway Connection

As mentioned earlier, our Lambda function is all set to interact with DynamoDB. While configuring permissions manually can be a bit of a chore, we have a convenient alternative. We can choose a Lambda Function blueprint that’s already prepped to connect with DynamoDB. The choice of whether to use this blueprint or not is entirely yours, but it serves as an invaluable reference point.

Screenshot explaining boiler plate code for Lambda Functions

Since we’re working with Python, let’s select the Python blueprint option.

Now, let’s delve into the Blueprint’s description. It promises a “backend microservice,” precisely what we need. It’s designed to interact seamlessly with DynamoDB and, most importantly, is primed to be triggered by an API Gateway call. A perfect match for our use case.

As we scroll down a bit on this page, we’re presented with the option to select an API Gateway trigger. Here, we decide whether to create a brand-new API Gateway or use an existing one. In this instance, we’re opting for a new API Gateway with the API Type set to HTTP and Security initially set to Open.

Screencap explaining the integration between Lambda Function and API Gateway

With these choices made, we can proceed to create our Lambda function.

Step 2: Injecting Code into the Lambda Function

Now, the spotlight shifts to the Lambda Function editor. Here, I’ve pasted the code we need. As mentioned earlier, we won’t delve into the nitty-gritty of the backend concept.

Here’s a vital callout: Our code involves not only interactions with DynamoDB but also subsequent calls to the Google Places API. We’ve taken a security-conscious approach and refrained from hardcoding any of our keys directly into the Lambda function. It’s a security risk we’d rather avoid. Instead, we’ve opted to store these keys, whether AWS or others, as environment variables. Lambda functions offer a powerful way to manage these keys as environment variables. You’ll find a screenshot below illustrating how to do this.

This configuration can be found under Configurations -> Environment variables.

Screencap explaining where to store the environment variables

If you’d like to explore the Lambda function’s code, it’s available in my GitHub Repository.

And here’s the exciting part: I’ve already run extensive tests on my Lambda Function using a sample test event, and it’s performing flawlessly.

Step 3: Configuring the API Gateway for Lambda Integration

With our Lambda function all set and performing beautifully, it’s time to venture into the API Gateway territory and fine-tune our configurations.

As you can see, the AWS Lambda function has magically created an API for us, thanks to our earlier choice of the predefined template. Now, let’s delve deeper into this API and make some crucial adjustments.

API which got automaitcally created while configuring Lambda Function

First, we’ll do a bit of housekeeping. I’ll go ahead and remove the existing route, making way for the one we truly desire. Our target API call will be: /api/v1/getallinfo.

Editing the default route

Once that’s done, I’ll save my changes and deploy them within the API Gateway.

Now, pay attention, this is where the magic happens. You’ll discover the base URL within the API Details section. It’s tucked away neatly inside the header titled “Invoke URL.”

Invoking the API gateway to trigger the lambda function

To put this to the test, I’ll append my desired route to the Invoke URL and fire up Postman. The moment of truth arrives, and voila! It’s working like a charm, returning precisely the output we anticipated.

API Call from Postman is successful in triggering the lambda function

So, give yourself a well-deserved pat on the back for making it this far in our journey.

Now, let’s tackle the elephant in the room: What if you decide to tweak the Lambda function’s code? Do you need to navigate the complex labyrinth of the AWS Console every time? The answer is a resounding No. We’re developers, and we work smart, not hard.

Let’s illustrate this point with a playful meme

Now, let’s explore how to make these updates in a more streamlined and efficient manner.

Step 4: Streamlining Code Updates with GitHub Actions and Workflows

Now, let’s dive into the world of GitHub Actions and workflows to supercharge our code management. Here’s the game plan:

• First, I’ll upload the code to my GitHub Repository, where our Lambda function resides. What’s the magic here? As soon as GitHub detects changes in the Lambda code within the master branch, it springs into action, triggering a carefully crafted workflow or GitHub Action.
• But here’s the twist: there’s no one-size-fits-all workflow for this on GitHub. We’re pioneers, so we’ll forge our own path. How? By creating our YAML file to define the workflow.
• I’ll start by navigating to my GitHub Repository, where I’ll upload the Lambda function code. Then, I’ll take things up a notch by creating a fresh folder named .github/workflows. Inside this digital realm, I’ll breathe life into our workflow through a YAML File.
• Now, here’s where the magic of AI assistance comes into play. While I may not be a DevOps wizard, I’ve enlisted the help of ChatGPT to conjure up a YAML file for me. It’s a piece of cake to use, and the code in the YAML file is as clear as day. If you ever find yourself pondering its intricacies, don’t hesitate to ask ChatGPT for a quick translation.

Below is the code of YAML File which ChatGPT generated for me

name: PlacesLambda

on:
  push:
    branches:
      - main

jobs:
  update-lambda:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Install AWS CLI
      run: |
        pip install awscli

    - name: Configure AWS CLI
      run: |
        aws configure set aws_access_key_id ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws configure set aws_secret_access_key ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws configure set region ${{ secrets.AWS_REGION }}
        
    - name: Create ZIP file
      run: |
        zip lambda_function.zip lambda_function.py
    
    - name: Update Lambda Function
      run: |
        aws lambda update-function-code --function-name ratingsInfo --zip-file fileb://lambda_function.zip

The code in YAML file should be clear enough for an experienced developer, however, just to explain it in very short, it is following the below steps

a) Before we dive into the details, let’s break down the YAML configuration file (.github/workflows/deploy.yml) that powers this automation. We've named our workflow "PlacesLambda," and it's triggered on every push to the main branch.

b) Job Setup: Inside our workflow, we have a single job named “update-lambda.” This job runs on the latest version of Ubuntu.

c) Code Checkout : The first action in our job is to check out the code from our GitHub repository using actions/checkout@v2.

d) Installing AWS CLI : To interact with AWS services, we need the AWS Command Line Interface (CLI). We install it using pip install awscli.

e) AWS CLI Configuration : To securely access our AWS resources, we configure the AWS CLI by setting our access key, secret access key, and the desired AWS region using secrets stored in GitHub.

f) Creating ZIP File : In this step, we create a ZIP file named lambda_function.zip. This ZIP file will contain the updated code for our Lambda function. We use the zip command to bundle our lambda_function.py file.

g) Update the lambda function : The final step is where the magic happens. We use the AWS CLI to update the code of our Lambda function named “ratingsInfo” with the contents of our ZIP file. This ensures that our AWS Lambda function always has the latest code whenever changes are pushed to the main branch of our GitHub repository.

A vital note, one that can’t be stressed enough: never expose those precious keys. Even in this scenario, where we’re making direct changes to the Lambda file from GitHub, we need AWS Keys to work our magic. But let’s keep them far from prying eyes. Instead, we’ll safeguard these keys within repo secrets, as showcased in the screenshot below

Screenshot explaining where should we be storing the secrets in Github Repository

Now, here’s where the excitement peaks. Anytime we make a tweak to our lambda_function.py from GitHub, the wheels of progress are set in motion. Like a well-oiled machine, it automatically updates the Lambda function code on AWS Servers. Cool, right?

But wait, there’s more. How do we make this process even cooler? What if a coding mishap leads to a compilation error or strays from the sacred path of Python syntax? Don’t fret; we’ve got you covered. We’ve enlisted the aid of a trusty GitHub Action known as PyLint. It’s the code guardian we all need. If your code falters with compilation errors or violates Python’s code of conduct, PyLint is unforgiving. It lowers your scores and, if necessary, declares your build a no-go.

Screenshot on how to set up PyLint for Python Code ( Lambda Function in this case )

And the best part? It’s a predefined template, ready for action. Just head to the GitHub Actions page, locate PyLint, and follow the on-screen instructions. Like a loyal ally, it’ll set up the workflow automatically, sparing you the trouble.

In case, your workflow fails, it gives you the information where and how it got failed . Interesting, isn’t it ?

Screenshot explaining the failure in workflows

So, with this robust setup in place, we’ve not only streamlined code updates but also fortified our code quality control. It’s a win-win, don’t you think?

If you enjoyed reading my content , don’t hesitate to connect on Linkedin

Sayonara, see you soon !