Serverless applications are becoming more and more popular and they are the barebones of event-driven architectures. The fact that we don’t need to manage infrastructure and we can just run code as a service is pretty awesome, it’s like orchestrating a symphony without knowing where the musicians sit, relying on invisible, ephemeral orchestrators that appear and disappear based on demand.

Back in 2014, AWS introduced Lambda, a serverless computing service that fits perfectly into this new paradigm, and it has been a game changer ever since. Lambda can run code on demand and has great integration with other AWS services, making it a perfect candidate for building serverless applications.

We can run so many languages such as NodeJS, Go, Python, Java, and many others. Some of the key components Lambda has are the following:

• Function Code: A block of code that runs in response to an event. It can be triggered by HTTP requests, changes in data, or even a schedule.
• Handler: The entry point of a Lambda function. It’s the function that Lambda calls when the function is invoked.
• Execution Role: An IAM role that grants permissions to the Lambda function. It defines what the function can do and what resources it can access.
• Configuration: The settings that define how the function behaves. It includes the function name, runtime, memory size, timeout, and other settings.

With these components, you can create a Lambda function that runs your code in response to events. You can deploy manually or using an Infrastructure as Code (IaC) tool like Terraform or CloudFormation.

One of the main challenges of building serverless applications is managing dependencies. Lambda already comes with some pre-installed packages, but if your code requires additional packages, you need to install them yourself. You can include these libraries by packaging them with your function code. I’m not a developer, but when I was learning about Lambda, I found that the process of packaging dependencies with the function code can be complicated, it took me some time to get it right and the documentation at the time didn’t seem clear enough to me, that’s why I decided to write this article — you can thank me later. ✌️

There are 2 main ways to package dependencies with your Lambda function code:

• Lambda Layers: You can create a Lambda Layer that contains the dependencies and attach it to the function. This is useful when you have common dependencies shared across multiple functions.
• Custom Packages: You can package the dependencies in a separate directory and include them with the function code. This is useful for large dependencies or when you have a lot of dependencies.

I’ll show you how to package custom dependencies with your Lambda function code using both methods in this article. I’ll use a simple code example to demonstrate the process, with a step-by-step guide that you can follow to package your dependencies.

I will use Python, but the same principles apply to other runtimes. Let’s say you have a Lambda function that uses the requests library to make HTTP requests. The requests library is not available by default in Lambda, so you need to include it with your function code.

Here’s how you can package the requests library with your Lambda function code:

1. Create a directory for your Lambda function code. You can name the directory lambda_function.

2. Create a requirements.txt file in the lambda_function directory. Add the following line to the requirements.txt file:

requests==2.31.0

3. Create a handler.py file in the lambda_function directory. Add the following code to the handler.py file:

import requests

def lambda_handler(event, context):
    response = requests.get('https://dog.ceo/api/breeds/list/all')
    return response.json()

This code accesses an API endpoint that provides a list of all dog breeds. When you make a GET request to this endpoint, it returns a JSON object containing a comprehensive list of all dog breeds, organized by their breed names. Each breed name may include sub-breeds as well. We are using the requests package to query this API.

The next step will be to install the requests package and create a zip:

# Install Requests
pip3 install -r requirements.txt -t .

# Create the Zip
zip -r handler.zip .

Now it’s time to deploy the Lambda function. This can be done manually using the AWS Management Console, or you can use an IaC tool like Terraform or CloudFormation to deploy the function.

When you deploy the function, you need to upload the lambda_function.zip file as the function code. The Lambda service will extract the contents of the zip file and run the function code.

Here’s an example with Terraform, if you want to know how to setup Terraform, check this tutorial here.

provider "aws" {
  region = "us-east-1"
}

resource "aws_iam_role" "role" {
  name = "example"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_lambda_function" "lambda_custom_package" {
  function_name = "lambda_custom_package"
  handler = "handler.lambda_handler"
  runtime = "python3.9"
  role = aws_iam_role.role.arn
  filename = "handler.zip"
}

To deploy the Terraform code, just run:

terraform init
terraform apply

This is how your function should look like:

If you invoke your function, you should get the result of the API call to the dog.ceo:

Another way to use custom packages is by creating a Lambda Layer, which is a way to share code and dependencies across multiple functions. You can create a Layer that contains the requests package and attach it to your function. This way, you can reuse the same layer across multiple functions, reducing the size of the function code and making it easier to manage dependencies.

Here’s how you can create a Lambda Layer using the requests package:

1. Create a directory named python for your Lambda Layer.

2. Create a requirements.txt file in the directory and add the following line to the requirements.txt file:

requests==2.31.0

3. The next step will be to install the requests package and create a zip:

# Install Requests
pip3 install -r requirements.txt -t .

# Create the Layer Zip
zip -r layer.zip python/

# Create the Code Zip
zip -r code.zip handler.py

Here’s the Terraform code with the Lambda layer, make sure to select the right Python version and also the architecture of your function:

provider "aws" {
  region = "us-east-1"
}

resource "aws_iam_role" "layer_role" {
  name = "layer_role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_lambda_layer_version" "lambda_layer" {
  layer_name = "requests_layer"
  filename = "layer.zip"
  compatible_runtimes = ["python3.9"]
  compatible_architectures = ["x86_64"]
}

resource "aws_lambda_function" "lambda_with_layer" {
  function_name = "lambda_with_layer"
  handler = "handler.lambda_handler"
  runtime = "python3.9"
  architectures = ["x86_64"]
  role = aws_iam_role.layer_role.arn
  filename = "code.zip"
  layers = [aws_lambda_layer_version.lambda_layer.arn]
}

To deploy the Terraform code, just run:

terraform init
terraform apply

This is how your function should look like:

There we go, now you know how to package custom dependencies with your Lambda function code. Everything we discussed here is available on my GitHub. This will help you manage dependencies more effectively and build serverless applications with ease.

I hope you found this article helpful, and I encourage you to explore Lambda further and experiment with different use cases. Serverless is a powerful tool that can help you build scalable, cost-effective applications, and Lambda is a key component of this paradigm. Happy coding! 🧑‍💻

Creating Lambda Custom Packages was originally published in Towards AWS on Medium, where people are continuing the conversation by highlighting and responding to this story.

Secrets are essential components of applications, including passwords, API keys, tokens, and certificates. If these secrets are compromised, your entire system could be at risk. That’s why it’s essential to apply the right security measures and to rotate them regularly.

In 2023, AI cybersecurity startup Lasso discovered over 1,600 valid Hugging Face API tokens exposed in code repositories, granting access to hundreds of organizations’ accounts. These tokens were exploited to access sensitive data and manipulate AI models, impacting organizations like Google, Meta, Microsoft, and VMware.

Consider your OpenAI API key. If exposed, unauthorized users could incur substantial charges by making requests on your behalf, manipulating your AI models, skewing data and outputs, or stealing sensitive information processed by your AI.

One of the main vectors of security breaches is the exposure of secrets. If a secret is leaked, it can be used to access sensitive data or perform unauthorized actions. To mitigate this risk, you should rotate your secrets regularly. This means generating new secrets and updating the applications that use them.

In this article, we’re diving into how you can rotate secrets on AWS using OpenTofu. For those not familiar, OpenTofu is the open-source and free version of Terraform, it’s a great tool to manage your infrastructure as code and it’s very easy to use.

If you’re tired of manually updating passwords and API keys or just want to step up your security game, stick around. We’re about to make secret rotation a breeze!

AWS Secrets Manager is a service that helps you to encrypt and centrally manage your secrets such as database credentials and API keys. With Secrets Manager, you can secure, audit, and manage secrets used to access your IT resources. By simply using Secrets Manager, you can avoid storing sensitive information in your code, and you can easily rotate secrets with built-in AWS Lambda functions.

You can also use AWS SSM Parameter Store to store secrets. However, AWS Secrets Manager provides additional features like automatic rotation, audit logs, and cross-account access. In this article, we’ll focus on AWS Secrets Manager.

By only using AWS Secret Manager you’re already taking a big step towards better security practices and meeting specific compliance requirements, frameworks such as ISO, SOC, and PCI among others, require managing secrets responsibly. But we can go even further by automating the rotation process. This way, you can ensure that your secrets are always up-to-date and that your applications are secure.

I will show you how to create a secret in AWS Secrets Manager and set up automatic rotation using Lambda, all deployed via OpenTofu. We will create a secret with a random API key and configure it to rotate every 30 days (You can customize the rotation period for your needs). We will also create an IAM role that allows the Secret Manager to rotate the secret on our behalf. Finally, we will deploy a Lambda function that will be triggered by Secrets Manager to perform the rotation. I also added to the stack an SNS topic so a notification can be sent once the secret is rotated.

You can find the Tofu code for this infrastructure in this GitHub repository. Here’s how the architecture looks like once deployed:

Let’s dive into the OpenTofu code that creates the infrastructure described above.

First, we need to create a secret in AWS Secrets Manager. I’m using a random API key as the secret value. I also create the rotation configuration for the secret, which specifies how often the secret should be rotated and the Lambda function that will perform the rotation.

# Create the Secret Manager
resource "aws_secretsmanager_secret" "apikey" {
  name = "apikey-${random_string.random.id}"
  description = "My API key"
  tags = {
    Name = "poc-apikey-${random_string.random.id}"
  }
}

# Store the API key in the secret
resource "aws_secretsmanager_secret_version" "apikey" {
  secret_id     = aws_secretsmanager_secret.apikey.id
  secret_string = var.apikey
}

# Create the Rotation Configuration
resource "aws_secretsmanager_secret_rotation" "rotation" {
  secret_id           = aws_secretsmanager_secret.apikey.id
  rotation_lambda_arn = aws_lambda_function.rotation_lambda.arn

  rotation_rules {
    automatically_after_days = 30
  }
}

Now we need to create the Lambda function that rotates the secret, important details are the environment variable SECRET_NAME which is the name of the “secret” in Secrets Manager, and the role that allows the Lambda function to update the secret. I will not go into details about the IAM role and policy, but you can find the code in the GitHub repository.

The same Lambda function can be used to rotate multiple secrets, in different AWS accounts or regions, as long as the Lambda function has the necessary permissions to update those secrets. You can also create multiple Lambda functions to rotate different secrets, depending on your needs.

# Create the Lambda Function to Rotate the Secret
resource "aws_lambda_function" "rotation_lambda" {
  filename         = data.archive_file.rotation_lambda_zip.output_path
  function_name    = "rotation-lambda-${random_string.random.id}"
  description      = "A lambda function to rotate the secret"
  architectures    = ["arm64"]
  role             = aws_iam_role.rotation_lambda_role.arn
  handler          = "handler.lambda_handler"
  runtime          = "python3.12"
  timeout          = 30
  memory_size      = 128
  source_code_hash = data.archive_file.rotation_lambda_zip.output_base64sha256
  environment {
    variables = {
      SECRET_NAME = aws_secretsmanager_secret.apikey.name
      TOPIC_ARN   = aws_sns_topic.sns_topic.arn
    }
  }
}

The Lambda function is written in Python and uses the AWS SDK to rotate the secret. It is a very simple code that generates a new API key and updates the secret in Secrets Manager, that should be enough to understand how it works. What you need to replace is the function create_secret (placeholder) for the code to pull/generate a new API key for your secret, once the Lambda function is triggered, we have in the handler the logic to update the secret in Secrets Manager.

import boto3
import os
import json
import uuid
from botocore.exceptions import ClientError

# Initialize the boto3 client
secret = boto3.client('secretsmanager')
sns = boto3.client('sns')

# Get environment variables
secret_name = os.environ['SECRET_NAME']
topic_arn = os.environ['TOPIC_ARN']

# Function to create a new secret value, replace by your own logic
def create_secret(secret_name):
    # Generate a new API key
    try:
        new_api_key = str(uuid.uuid4())
        secret = {
        "api_key": new_api_key
        }
        return json.dumps(secret)
    except Exception as e:
        print(f"Failed to generate a new secret value: {e}")

# Function to format the message log
def format_response(secret_name, response):
    formatted_response = {
    "RequestId": response['ResponseMetadata']['RequestId'],
    "Date": response['ResponseMetadata']['HTTPHeaders']['date'],
    "SecretId": secret_name,
    "Status": "Success" if response['ResponseMetadata']['HTTPStatusCode'] == 200 else "Failed",
    "HTTPStatusCode": response['ResponseMetadata']['HTTPStatusCode'],        
    }
    return formatted_response

# Send a notification to the SNS topic
def send_notification(message):
    try:
        response = sns.publish(
            TopicArn=topic_arn,
            Message=json.dumps(message),
            Subject='Secret rotation completed'
        )
        print(f"Notification sent to {topic_arn}")
        return response
    except ClientError as e:
        print(f"Failed to send notification to {topic_arn}: {e}")

def lambda_handler(event, context):

    # Create a new secret
    new_secret = create_secret(secret_name)

    # Update the secret value
    try:       
        response = secret.put_secret_value(SecretId=secret_name, SecretString=new_secret)
        formatted_response_json = format_response(secret_name, response)
        print(f"Secret rotation completed")
        print(formatted_response_json)
        sns_notification = send_notification(formatted_response_json)
        print(sns_notification)
    except ClientError as e:
        print(f"Failed to update secret {secret_name}: {e}")
        return {
            "statusCode": 500,
            "body": json.dumps("Secret rotation failed")
        }
    
    # Return a success message
    return {
        "statusCode": 200,
        "body": json.dumps("Secret rotation completed")
    }

Although it’s possible also to use the update_secret method in AWS Secrets Manager to rotate the secret, I recommend using put_secret_value because it only updates the secret value and doesn’t touch the metadata, such as SecretString, SecretBinary, Description, KmsKeyId, and Tags. This way, you can keep the metadata intact and only rotate the secret value.

You can check the cloudwatch logs to see if the Lambda ran correctly and in the secret manager, you can retrieve the secret to know if it was changed.

As you can see below you can also manually trigger the rotation in the secret manager, under Rotation > Rotate secret immediately.

After rotating the secret, the Lambda will also publish a notification to an SNS topic. From the SNS you can subscribe to your email address or even other AWS resources, this will allow for a better monitoring of the rotation task:

As easy as it looks, this is a very strong security practice that you can implement in your infrastructure. It’s a simple way to keep your secrets up-to-date and secure. By automating the rotation process, you can ensure that your applications are always using the latest secrets and that your system is protected from unauthorized access.

We have the habit of putting every security practice in security tools, but the tools only make it easier to implement these practices. You can buy a supercar, but it doesn’t do much if you don’t know how to drive. It’s crucial to understand the practice to use the tool properly and make the most of it.

As we saw in this article, it’s not hard to implement secret rotation and it’s a simple yet powerful way to improve the security of your applications.

I hope this article has been helpful to you. I would love to answer your questions and feedback, you’re welcome to leave a comment below.

References:

https://www.securityweek.com/major-organizations-using-hugging-face-ai-tools-put-at-risk-by-leaked-api-tokens/

https://aws.amazon.com/secrets-manager/

https://opentofu.org/

Automate AWS Secret Rotation was originally published in Towards AWS on Medium, where people are continuing the conversation by highlighting and responding to this story.

Infrastructure-as-a-Code technology has been used as the de facto way to build infrastructure in the cloud, especially by CI/CD automation tools such as Jenkins, GitHub Actions and so many others. IaC allows us to make our applications more dynamic by giving the ability to deploy applications or “Stacks” multiple times per day.

I’ll link an article explaining deeper this concept here from my friend Fernando Cardoso:

Putting Security into The IaC Pipeline

There is no question that these tools provide great flexibility and resources to empower developers to make more with less, however…

How to measure security in these configurations and what kind of baselines should we follow?

A: These are the concerns that Trend Micro Cloud One Conformity is helping customers to solve.

Trend Micro Cloud One Conformity is a great tool to help you avoid misconfiguration to be introduced in your cloud environment. The platform analyzes your cloud environment looking for misconfigurations/security risks and notifying when someone (Human) or something (CI/CD tools) introduces a new configuration out of best practices in different phases of your development cycle. Theses misconfigurations can be detected in the build phase when the team is changing the code by analyzing your IaC templates directly from the IDE (VS Code) or even during the build time by your CI/CD tool. At the runtime of the application, the platform monitors the cloud accounts for any manual intervention or new misconfiguration introduced directly to the cloud environment. Look the illustration below that describes where the platform fits in the DevOps cycle:

With the mission to make it easier for customers to scan their IaC templates, we've decided to create this open-source project so every customer can have access to the code and use it on their own. The project is in GitHub and together with the code we also created a GitHub Action that is a CI/CD automation tool to make this scan be reproducible, you can get this Action on the GitHub Marketplace for FREE and use in any GitHub repository, check it out:

Cloud One Conformity Pipeline Scanner - GitHub Marketplace

The Action that we created in GitHub uses an NPM module to implement all the functionality that the Action needs.

Raphael Bottino originally created the Node.js module that translates the Cloud One Conformity API's to make it easier to use in a pipeline and in the IDE to scan your IaC templates, you can check the NPM module here and use in your own projects as well:

cloud-conformity

We're not going to cover the IDE integration in this article, but it's already well explored here in this another article below, we highly recommend you to read:

Shift Well-Architecture Left. By Extension, Security Will Follow

The best way to demonstrate why all this tech is awesome is to get our hands dirty, so let's try out this Action!

I've created a repository in GitHub to show you how this Action work, you can have access to the repository in this link below:

felipecosta09/Demo-Cloud-One

I've added a Cloud Formation template to this repository that simulates a real scenario of a customer getting an infrastructure deployed in AWS Cloud, this is the Cloud Formation template. We also created the pipeline file that GitHub Actions require to automate your build and every time that there is a new commit, your template will be scanned.

The Pipeline has only 2 actions, the first action will checkout the code from the repository and the second will scan the template, below is the pipeline file I commented the code also make it easier to read.

Under the Cloud Formation Action, you can see that there are some parameters that need to be passed, they're:

• Conformity API Key - The form of authentication required to access the API, you can see how you generate the API Key here;
• Triggers - How many misconfigurations are acceptable based on the severity (maxExtreme, maxVeryHigh, maxHigh, maxMedium and maxLow);
• Cloud Conformity Regions - The region where your console is located;
• Template Path - The path of the template that you want to scan;

Based on the "severity triggers" you pipeline will go through or fail, and there it is: you'll have a fully automated tool that can analyze IaC templates and help you shift-left security in your development cycle.

It's amazing how using a simple tool can make such a great difference in how we approach security by helping developers to analyze their code directly in the current toolset. But we still have a gap here, because if you don't use GitHub actions and decide to use a different CI/CD tool such as Gitlab or even Jenkins, you won't be able to execute the steps that I just showed. That's why we've decided to port this project to run inside a docker container, which makes it very portable and keeps the lightweight execution. We're always looking to provide as many options as we can for our customers to implement security controls. Take a look at the container that is published in Dockerhub.

This is the command that you can use to scan your templates using a simple docker command:

docker run -v /home/ec2-user/dynamotest.template:/app/dynamotest.template -e cc_apikey=$MYAPIKEY -e cc_region=$MYREGION -e maxExtreme=0 -e maxVeryHigh=0 -e maxHigh=0 -e maxMedium=0 -e maxLow=0 -e templatePath=infrastructure.yaml felipecosta09/conformity-template-scanner-pipeline:latest

PS.: To be able to scan a local template from a machine or inside a pipeline, the parameter “-v” is required in the docker run command, the example specifies a local file being copied to the container that will scan the Cloud Formation template /home/ec2user/dynamotest.template:/app/infrastructure.yaml, where:

• /home/ec2-user/infrastructure.yaml — Represent the absolute path of the local Cloud Formation template file to be scanned;
• /app/infrastructure.yaml — The path where the file will be copied (ONLY CHANGE THE FILE NAME OF THE TEMPLATE);

You can also invoke the JS file within the implementation of the npm package to scan your templates, just clone the original GitHub Repo, replace the variables and of course, you need nodejs to be installed, take a look:

cc_apikey=$MYAPIKEY cc_region=$MYREGION maxExtreme=0 maxVeryHigh=0 maxHigh=0 maxMedium=0 maxLow=0 templatePath=infrastructure.yaml node scan.js

We hope that these tools will help you in your journey to the cloud, application modernization, and to achieve operational excellence. All the work that we start a couple of months ago with this project was because we had so many feedbacks from customers and also in the open-source community, so if there is something that we're missing, please give your feedback so we can get even better. Our mission at the end of the day is to help with security challenges so you can be successful.

References:

https://www.npmjs.com/package/cloud-conformity

https://github.com/raphabot/cloud-conformity-cloudformation-scanner

https://hub.docker.com/r/raphabot/conformity-template-scanner-pipeline

https://github.com/raphabot/conformity-template-scanner-pipeline

https://github.com/cloudconformity/documentation-api

https://www.cloudconformity.com/

https://github.com/felipecosta09/Demo-Cloud-One

https://medium.com/swlh/putting-security-into-the-iac-pipeline-4de98f88ad24

https://medium.com/@raphabot/shift-well-architecture-left-by-extension-security-will-follow-9012168b56e8

Nowadays the CI/CD process is essential to ensure business agility and therefore the success of many companies. DevOps teams have been changing their approach to application development, with a cloud-native and API-first mentality. As a result, teams can now use multiples tools and deploy applications in different environments. Changing deployment or adding more integrations is not a problem anymore and allows the application to take advantage of many services and/or resources that cloud providers have to offer.

Securing CI/CD pipelines is an obvious requirement since it is a top priority for DevOps teams today. Accomplishing that, however, is not a straightforward process. Implementing frictionless security early in the CI/CD workflow is the best approach to “shift-left” the responsibility for securing any application: the earlier your team can identify new weaknesses or vulnerabilities inside the applications, the easier it will be to fix them. Waiting until after the application is deployed to production — at runtime — makes it a lot more difficult to implement any changes.

The following tools will help your DevOps teams to have more visibility on the weakness in their code and help also to prioritize security vulnerabilities that matter the most, spending less time fixing non-critical bugs to reduce the overall risk of your applications.

Here is an article from my friend Fernando Cardoso that explains security tools for DevOps pipelines in-depth:

How to integrate security on the DevOps pipeline?

The best way to understand how these concepts are applied in real life is to get your hands dirty, so let’s create some containers and scan them in pipeline time.

The architecture is pretty simple: we have a code repository, a CI/CD build system, a container registry, and a platform to deploy the application (I will not cover deployment in this article, maybe in a future one 🤞). We’ll build the container from scratch, push to a container registry, and scan this container to check how many security issues I or my team introduced to this new application.

For starters, I will use GitHub as my code repository and GitHub Actions as my CI/CD system because it is integrated and so easy to use, and to scan my brand new container application I will use Trend Micro Cloud One™Container Security (Formerly Deep Security Smart Check), as it allows me to make sure that before I deploy any application to my production environment, I can check for any Malware, Vulnerabilities, Secrets, and Compliance and block my pipeline to go further in case I find any issues with my code.

PS.: I’m also using AWS ECR as my container register, but the same process applies to Microsoft® Azure, Google Cloud, and any other cloud provider with a Docker Register base or even local environment. 👍

I’ve created a public repository on my GitHub called Trend-Micro-Container-Security-for-CI-CD. In this repository, I have a Dockerfile that I use to build my container and a few other files that I need for my application to work. I will start by using Github Action to run my first build:

felipecosta09/Trend-Micro-Container-Security-for-CI-CD

To create your first Github Actions pipeline you need to create a folder structure inside your repository in this format:

My-Repo/.github/workflow/pipeline.yml

Inside the workflow folder, you need to create a YML file that will represent your pipeline, I already created one in my repo:

PS.: You may have noticed that some of the environment variables are secrets, is just to make it more readable and secure to expose on the internet. You should always use secrets to expose your keys and secrets.

The tasks to Build, Tag, and Push containers are already known by everyone working with CI/CD pipelines, but I want to focus on how you can scan these containers as soon as they’re ready. The last part of the pipeline is where we scan the container, is a simple GitHub Action that I’ve created and published to GitHub Marketplace so you can use to easily scan your containers without spending a lot of time creating code and most importantly, you can block the pipeline from going further in case it reaches the threshold defined by you. To use the Action, you just select from the Marketplace, input the information requested, and it’s done!

The mechanism to scan the container is based on a Docker container that the Trend Micro team has created to use Cloud One™ Container Security (Formerly Deep Security Smart Check) API to start and check the status of the scans. This means that you can scan containers in any pipeline, not only in GitHub Actions. I’ll leave the link to the Action in the marketplace, so you can learn how to use it, what kind of inputs you’ll need to provide, and what kind of results you should expect. I also created examples on how to use this Action with Azure and AWS container registries and even using a Docker command:

Deep Security Smart Check Scan Action - GitHub Marketplace

To add the Action to your pipeline you can copy the code example from the marketplace or you can edit your pipeline YML code directly from the GitHub website and you’ll be able to search for marketplace actions.

felipecosta09/Deep-Security-Smart-Check-Scan-Action

Back to our pipeline, as soon as you run the pipeline and provide the required information, the Action should start to Build, Tag, Push, and Scan the container. In our case, the second job “Container_Scan” failed the pipeline because after the container scan completed, we identified:

• 1 Malware Found (Malicious File)
• 1 Secret Exposed (Private key)
• 156 Vulnerabilities (Including OS packages and 3rd party packages)
• 13 Checklists (We failed to be compliant with PCI-DSS v3, HIPAA and NIST 800–190)

I detailed also the information that you can see in the Smart Check console, including the Overall Scan Details and each one of the layers in which Smart Check identified issues with our code:

In case you’re running a Java container application you’ll also count with a dependency check, through a partnership with Snyk, which is the most advanced and accurate Open-Source vulnerability database. Below there is an example on how it will look like in Smart Check console:

You can easily go to Snyk KB to check more about each one of the vulnerabilities and create Pull Request to each one of them.

Snyk - Deserialization of Untrusted Data in com.fasterxml.jackson.core:jackson-databind

Imagine how many vulnerabilities/weaknesses my team or I could introduce to a new or old application. Without using these tools, we would probably never know. The fact that we can detect and prevent these issues from being deployed to a production application is one of the best security practices that you can follow. The more you “shift-left” the security responsibilities, the more you’ll increase the reliability and decrease the overall security risk.

References:

https://www.github.com/deep-security/smartcheck-scan-action

https://www.trendmicro.com/containers

https://www.github.com/felipecosta09/Deep-Security-Smart-Check-Scan-Action

https://github.com/marketplace/actions/cloud-one-container-security-scan-action

https://snyk.io/

KOPS (Kubernetes Operations) helps you to create, destroy, upgrade and maintain production-grade, highly available and scalable Kubernetes clusters from the command line. It is the easiest way to get a production Kubernetes cluster up and running.

This article is meant to help you create your self-managed K8s cluster, it’s long but very easy to follow. The official documentation for KOPS (Kubernetes Operations) can be found here:

GitHub - kubernetes/kops: Kubernetes Operations (kOps) - Production Grade k8s Installation, Upgrades and Management

I’m using AWS as a preferred cloud provider to run this cluster. This tool also supports Google Cloud, OpenStack, VMware vSphere in alpha, and other platforms planned.

KOPS will create several different resources in your AWS account to set up the K8s cluster, such as EC2 instances, Route 53 entries, Autoscaling Groups, and Load Balancers to provide high-availability and scalability to your cluster.

To start, we will need to have an EC2 instance that can be an instance — that you already own or you can set up a new one and assign the right permissions to this instance, to set up a new one, follow the manual below:

Set up to use Amazon EC2

It is also required to have these IAM permissions below so the EC2 can set up your cluster:

AmazonEC2FullAccess
AmazonRoute53FullAccess
AmazonS3FullAccess
IAMFullAccess
AmazonVPCFullAccess

To set up these permissions you can create an IAM user, set up the permissions and use AWS CLI to configure the Access Key and Secret, so your EC2 instance can use KOPS to set up the cluster. You can check in the official KOPS manual to configure that:

kubernetes/kops

The other option is to use IAM Roles and attach these permissions to your EC2 instance — as I did — and you should have the same results either way. To create an IAM Role, follow this manual:

IAM Roles for Amazon EC2

Next step will be to install the 2 required tools to set up your cluster:

Install Kubectl:

Install KOPS:

PS.: You’ll also need the AWS CLI to be installed — since I’m using an Amazon Linux 2 machine, AWS CLI is already installed.

After installing the tools, we need to create an S3 bucket so can store the state of your cluster, to create the bucket follow the configurations below:

PS.: This is the most basic way to set up your S3 bucket. I also recommend you set up versioning your S3 bucket in case you ever need to revert or recover a previous state store and also default encryption to your bucket, but it’s optional

I’ll also set up some environment variables to make easier to execute commands in the future:

PS.: You can choose any name you want but the name MUST END with “k8s.local” due to DNS configurations that the KOPS will execute during the process to create the cluster. Also, your S3 bucket name needs to be unique across the AWS Cloud.

You will need to choose which availabilities zones are available for your cluster to use. In this example, we will be deploying our cluster to the us-east-1 region, to check the availability zones to that region, just follow the options below:

You can check out all the Regions and Availability codes in this link below:

Regions, Availability Zones, and Local Zones

In my case, I’m creating my nodes each one on a different availability zone (1 Master and 3 Worker Nodes). By executing the command below you’ll create the configuration in the S3 bucket so KOPS can read and deploy your cluster:

The above command will generate a cluster configuration, but not start building it yet. As per the message, you must set up an ssh public key and add this key to the cluster:

PS.: Skip the passphrase configuration

I’ll run the cluster within the master default configuration, but if you want to modify the cluster set up configurations, you can use the following command:

I modified the node’s configuration on the cluster so can scale more instances than the default, you can change instance type and minimum and maximum instances by using the following command and editing the variables:

If you want to double-check the configurations that KOPS will apply, you can use the following command:

To apply all the configuration we’ve done so far, execute the command below to start creating your K8s cluster, based on the config files that we store in the S3 Bucket:

It will take time to create the cluster (usually from 10 to 15 minutes) so be patient. You can check the status of the cluster creation by executing the following:

PS.: The command above will show several errors during the process but don’t worry, this happens because the cluster is not ready yet.

After a couple of minutes when the cluster is 100% operational, you’ll be able to check your nodes and master by executing the same command as before:

Done! You have just created an enterprise-grade K8s cluster in the cloud.

If you’re worried about the cost of this setup, I made an estimation cost using AWS Pricing Calculator, generally speaking, this set up will cost you around 0.25USD per hour. You can check the full estimate cost in the link below:

https://calculator.aws/#/estimate?id=806bcedd844ac67481c9e3407ef5003fec504e20

Remember to delete your cluster if you’re not using it, to delete your cluster, just execute the command below:

References:

https://github.com/kubernetes/kops

https://kubernetes.io/

https://calculator.aws/