Image by Mohamed Hassan from Pixabay

GitLeaks: A Secret Scanner Tool

In DevOps processes, automating the delivery of code changes and new features to end user seamlessly and frequently, with CI process where testing and build processes are carried out, and then the CD process where the deployment process is performed, has provided many advantages, but what about security?

“I wonder if I applied security best practice?” Isn’t that question a source of concern for many people in the field? If this isn’t a concern, it should be.

Because attacks on applications cause great financial and reputational damage to companies, and the number and type of attacks are increasing every day. And companies have realized that their expenses for application security are much less compared to the compensation they pay after attacks.

Here is the top ten of OWASP:

Isn’t performing security checks at the last stage of the application development process a step that both slows things down and is late? It would make more sense to implement security checks at every stage in the SDLC and automate them in the CI/CD pipeline.

In fact, it is already possible to take the DevOps process, which starts after the code commit, to the pre-commit stage with DevSecOps. Let’s briefly look at how this is done with GitLeaks tool, one of the secret scanning tools that scans source code and detects hard-coded secrets. GitLeaks is a fast, light-weight, portable, and open-source secret scanner for git repositories, files, and directories.

First of all, it is necessary to install GitLeaks, but thanks to Docker, we will avoid installing it manually depending on the OS. We are cloning OWASP’s Juice Shop application (1) in an Ubuntu EC2 instance with Docker installed.

Then create a file called pre-commit in the hooks folder in .git (hidden folder) and enter the following commands into it:

docker pull zricethezav/gitleaks:latest
export path_to_host_folder_to_scan=/home/ubuntu/juice-shop
docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest detect --source="/path" 

Details of the above commands are available in the documentation of the zricethezav/gitleaks image on dockerhub. With the detect command, hard-coded secrets in the application at the /home/ubuntu/juice-shop path can be detected right before being pushed to the remote repository by the developer:

( If you use — verbose, you will get detailed output. However, the output above is deliberately simplified, I did not use — verbose for simplicity)

If you see that output you can be sure that you could not git push your code changes. Fine…

But of course, you should not rely on developer. It is necessary to automate this work as a DevSecOps engineer and reflect it in the pipeline:

variables:
    IMAGE_NAME: tariksk/demo-app
    IMAGE_TAG: juice-shop-1.1

stages:
    - cache
    - test
    - build

create_cache:
    stage: cache
    image: node:18-bullseye
    script:
        - yarn install
    cache: 
        key: 
            files:
                - yarn.lock
        paths:
            - node_modules/
            - yarn.lock
            - .yarn
        policy: pull-push
        
yarn_test:
    image: node:18-bullseye
    stage: test
    script:
        - yarn install
        - yarn test
    cache: 
        key: 
            files:
                - yarn.lock
        paths:
            - node_modules/
            - yarn.lock
            - .yarn
        policy: pull

gitleaks:
    stage: test
    image:
        name: zricethezav/gitleaks
        entrypoint: [""]
    script:
        - gitleaks detect --verbose --source .


build_image:
    stage: build
    image: docker:24
    services:
        - docker:24-dind
    before_script:
        - echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin
    script:
        - docker build -t $IMAGE_NAME:$IMAGE_TAG .
        - docker push $IMAGE_NAME:$IMAGE_TAG

According to this pipeline (.gitlab-ci.yml), if GitLeaks detects a secret in the source code, the pipeline will fail and the docker image cannot be built (next job will be skipped).

But the important point is this: The Gitleaks job which is performed in the pipeline takes place after the code commit, that is, after the developer pushes the relevant codes to the remote repository, that is, after it is too late!

Why is it late? Because even if you remove sensitive data later, the history in the repository remains intact! Therefore, using pre-commit hook is a more secure option.

Notes:

(1) OWASP Juice Shop is probably the most modern and sophisticated insecure web application! It can be used in security trainings, awareness demos, CTFs and as a guinea pig for security tools! Juice Shop encompasses vulnerabilities from the entire OWASP Top Ten along with many other security flaws found in real-world applications!