Monitor git secrets on Azure DevOps with Gitleaks

Introduction

In this article we will see how to scan all repos of an Azure DevOps project.

The secret scanning solution will respect the following guidelines:

• 1 single DevOps pipeline per DevOps project that launches Gitleaks and sends its scan results to an Azure Application Insights.
• This pipeline can be triggered manually for a spot check or through any supported scheduled triggers provided by Microsoft.
• Developers are able to declare a secret as a false positive following Gitleaks procedure.
• 1 Azure monitoring Workbook where you can find:

1. An overview tab that displays the total number of secrets found.
2. A “Secrets” tab which gives more metrics such as
3. the secret trends -> number of secrets observed during each scan,
4. count of secrets per repository,
5. a complete inventory of results exportable to Excel.

Prerequisite

• Disable the option “Protect access to repositories in YAML pipelines” to allow the Azure DevOps Build Service to pull remote repositories. You can find this option by navigating to the project settings page under “Pipeline” > “Settings”.
• Grant version control permissions to the build service as described in the following procedure.
• You have an Azure Application Insights and are able to read its connection string.

Pipeline

Create an Azure DevOps repo following this procedure and add on this repo the 3 following files (all the source code is available here: azure-devops-secret-scanning).

• azure-pipelines.yml → This yaml file describes the Azure DevOps pipeline workflow that will perform the secret scan.
• send_az_appinsights_event_telemetry.sh → This bash script is in charge of sending data into Azure Application Insights. I have posted a dedicated article here focused on this script and its usages.
• scan_azure_devops_secret.sh → This bash script performs the secret scanning with Gitleaks.

You can now create the Azure DevOps pipeline based on the yaml file you just pushed, for more details on this procedure you can consult the following article.

You can now run the pipeline manually from Azure DevOps by providing the following information:

• Your Azure Application Insights connection string.
• A valid email and name of a git user in your organisation that will be in charge of the secret scanning.

Monitoring Workbook

We can now publish the Azure Monitoring Workbook to find:

• An overview tab that displays the total number of secrets found.

• A “Secrets” tab which gives more metrics such as

1. the secret trends -> number of secrets observed during each scan,
2. count of secrets per repository,
3. a complete inventory of results exportable to Excel.

You can publish the Azure Monitoring workbook following this procedure “Creating a new workbook” and by copy pasting the json code available here: azure-devops-secret-scanning/workbook.json.

Conclusion

Exposed credentials in engineering systems provide easily exploitable opportunities for attackers. To defend against this threat, we demonstrate here how to use a free monitoring workflow that prevents any credentials from being leaked.

Feel free to use it, share it and improve it.

See You in the Cloud

Jamesdld