Integrating Custom YAML Policies in Checkov with Jenkins and Slack

Description

Discover how to effortlessly manage compliance by creating and integrating custom YAML policies in Checkov with Jenkins and Slack. This guide makes it easy to ensure your infrastructure meets security and compliance standards.

Introduction:

• Checkov is a static code analysis tool designed to identify security vulnerabilities in your code.
• It offers the flexibility to create custom policies using Python and YAML to suit specific security requirements.
• By default, Checkov includes 750 policies for comprehensive code analysis.
• Checkov supports a wide range of frameworks, including Terraform, CloudFormation, Kubernetes, and more, ensuring compatibility with various development environments.
• The tool generates detailed reports and recommendations for addressing detected failed policies, facilitating proactive security measures.

Prerequisites:

1. Jenkins server integrated with Github for version control management and Slack for real time communication.
2. Ensure you have Checkov and Terraform installed in your Jenkins server.
3. Install Checkov by running this command.

pip3 install checkov

or 

brew install checkov

Step 1: Terraform Files

Firstly we will create our terraform code which we’ll need for our checkov process. We will start off by creating a simple aws instance. Include a providers.tf file in there also.

Main.tf 

resource "aws_instance" "main" {
  ami           = "ami-xxxxxx"
  instance_type = "t2.micro"

  root_block_device {
    volume_size = 25
    volume_type = "io2"

  }
  tags = {
    Name  = "main"
    owner = "Umid"
  }
}

providers.tf


provider "aws" {
  region = "us-east-1"
}

Step 2: Yaml Custom Policy Structure

Before creating custom Checkov policies, it’s important to understand the format and structure they follow. Each custom policy includes metadata, including name, ID, and category.

1. Metadata:
2. Policy Name
3. ID — Following the format CKV2_<provider>_<number>
4. Category — Choose from predefined values:

• GENERAL_SECURITY
• LOGGING
• ENCRYPTION
• NETWORKING
• IAM
• BACKUP_AND_RECOVERY
• CONVENTION
• SECRETS
• KUBERNETES
• APPLICATION_SECURITY
• SUPPLY_CHAIN
• API_SECURITY

Step 3: Definition Block

The definition block serves as the core component where you construct the logic of your policy. Below is the fundamental structure of definition blocks:

Definition Block(s) — either Attribute Block(s) or Connection State Block(s) or both

• Logical Operator(s) (optional)
• Filter (optional)

For detailed insights into creating definition blocks, I highly recommend reading the official documentation available at: Checkov Custom Policies — YAML Custom Policies

Step 4: Our First Custom Policy

To begin our policy creation process, we’ll create our first YAML file containing a custom policy. Our policy objective is to ensure that our EC2 instance Terraform code includes the specification of an “availability_zone”.

---
metadata:
  name: "Check ec2 instance has availability zone"
  id: "CKV2_AWS_1001"
  category: "GENERAL_SECURITY"
definition:
  cond_type: "attribute"
  resource_type:
    -  "aws_instance"
  attribute: "availability_zone"
  operator: "exists"

Step 5: Second Custom policy

For our second custom policy, we’ll continue by creating another YAML file. This policy aims to enforce the “instance_type” parameter in our EC2 Terraform code.

---
metadata:
  name: "ec2 instance requires instance type"
  id: "CKV2_AWS_1002"
  category: "GENERAL SECURITY"
definition:
  cond_type: "attribute"
  resource_type:
    -  "aws_instance"
  attribute: "instance_type"
  operator: "exists"

Step 6: Third Custom policy

To continue our policy development journey, we’ll create another YAML file for our third custom policy. This policy aims to mandate the inclusion of an “Owner” key in the tags of our EC2 Terraform code.

---
metadata:
  id: "CKV2_AWS_1003"
  name: "Ensure EC2 instances have an 'owner' tag with a value"
  category: "GENERAL_SECURITY"
definition:
  cond_type: "attribute"
  resource_type:
    - "aws_instance"
  attribute: "tags.owner"
  operator: "exists"

Step 7: Fourth Custom Policy

For our fourth custom policy, let’s continue by crafting another YAML file. This policy focuses on ensuring the presence of the “Name” key in the tags section of our EC2 Terraform code.

---
metadata:
  name: "Check that you have 'Name' in ec2 tags"
  id: "CKV2_AWS_1004"
  category: "GENERAL_SECURITY"
definition:
  cond_type: "attribute"
  resource_type:
    - "aws_instance"
  attribute: "tags.Name"
  operator: "exists"

Step 8: Fifth Custom policy

For our final custom policy, let’s proceed by creating another YAML file. This policy’s objective is to enforce the selection of one of the four specified options for the EBS volume type in our Terraform code.

---
metadata:
  id: "CKV2_AWS_1005"
  name: "Ensure EC2 Instances have a volume type of standard, gp2, gp3, and io1"
  category: "GENERAL_SECURITY"
definition:
    cond_type: "attribute"
    resource_types:
      - "aws_instance"
    attribute: "root_block_device.volume_type"
    operator: "subset"
    value:
      - "standard"
      - "gp2"
      - "gp3"
      - "io1"

Step 9: Testing our policies against our Terraform code

Before running our Checkov commands, it’s important to follow best practices. We have to create a file from the Terraform plan. Then Execute our checkov command against the new file. This ensures a thorough and accurate evaluation of your infrastructure against your custom policies.

Organize your directories and files as follows:

1. Initialize Terraform by running

terraform init

2. Generate a Terraform plan file:

terraform plan -out tfplan

3. Convert the Terraform plan to JSON format:

terraform show -json tfplan | jq '.' > tfplan.json

4. Execute the Checkov command against the generated JSON file, specifying custom policies:

checkov -f tfplan.json --external-checks-dir custom_policies/ --check CKV2_AWS_1001,CKV2_AWS_1002,CKV2_AWS_1003,CKV2_AWS_1004,CKV2_AWS_1005

After we execute this command and run it against our tfplan.json file we will get this output:

As you can see our Terraform code has 3 success checks and 2 fails.

Step 10: Jenkinsfile

Now that we have an understanding of how Checkov outputs its policy reports, let’s create a Jenkinsfile. We’ll execute these same steps, but with some additional ones, such as creating a file from the output of the Checkov command and sending a Slack message if there are failed checks.

pipeline {
    agent any
    environment {
        SECRET_KEY = credentials('slack-token')
    }
    stages {
        stage('Initialize') {
            steps {
                script {
                    sh 'terraform init'
                }
            }
        }

        stage('Create Terraform Plan') {
            steps {
                script {
                    sh 'terraform plan -out tfplan'
                }
            }
        }

        stage('Convert Plan to JSON') {
            steps {
                script {
                    sh 'terraform show -json tfplan > tfplan.json'
                }
            }
        }

        stage('Check Compliance') {
            steps {
                script {
                    sh 'checkov -f tfplan.json --external-checks-dir custom_policies/ --check CKV2_AWS_1001,CKV2_AWS_1002,CKV2_AWS_1003,CKV2_AWS_1004,CKV2_AWS_1005 > checkov_output.txt || true'
                    sh 'cat checkov_output.txt'
                    def failedChecks = sh(script: 'grep -o "Failed checks: [1-9]*" checkov_output.txt | grep -o "[1-9]*" || true', returnStdout: true).trim()

                    if (failedChecks) {
                    slackSend(channel: "checkov", message: "There are $failedChecks failed policies in your code, please review and update", sendAsText: true)
                    } else {
                        echo "All checks passed. Proceeding to apply changes."
                        sh 'terraform apply tfplan'
                    }
                }
            }
        }
    }
}

After you create a Jenkinsfile, push to Github. Make sure you have Slack and Github integrated with Jenkins.

Step 11: Jenkins Pipeline Run

Executed the Jenkins pipeline successfully, triggering a Slack message to our designated channel.

Step 12: Slack Message

Receive a Slack notification in our designated channel.

CONCLUSION

In conclusion, incorporating predeployment checks into our CI/CD pipeline is crucial for ensuring application reliability and safety. Tools like Checkov are invaluable for automating scans and identifying security vulnerabilities early on. By integrating Checkov, we can proactively address issues before deployment, reducing the risk of costly incidents and ensuring compliance with best practices.

https://github.com/Umidbekakumo/checkov-custompolicies-slack