Streamlined Workflows with GitHub Actions, Dockerfile Integration, and Fetching Secrets from HashiCorp Vault
Introduction:
GitHub Actions is a powerful workflow automation tool provided by GitHub. It allows developers to define custom workflows that automate various tasks related to their software development processes. These workflows are defined using YAML files, which specify a series of steps to be executed in response to specific events, such as code pushes, pull requests, or scheduled triggers.
Each workflow consists of one or more jobs, and each job runs on a specific type of runner environment, such as Ubuntu, macOS, or Windows. Jobs are composed of individual steps, which are sequential actions executed within the runner environment. Steps can be predefined actions provided by GitHub, custom scripts, or a combination of both.
GitHub Actions offers a wide range of predefined actions that cover common tasks like checking out the source code, building and testing applications, deploying artifacts, interacting with external services, and more. These actions can be easily incorporated into workflows, reducing the need for manual intervention and enabling automated software development processes.
Workflows can be triggered by events like code pushes, pull requests, issue comments, or even on a schedule. This allows developers to automate tasks like running tests, building and deploying applications, updating documentation, and notifying team members about changes or issues.
GitHub Actions also provides powerful features like environment variables, secrets management for securely storing sensitive information, and the ability to run workflows on self-hosted runners for more control over the execution environment.
By leveraging GitHub Actions workflows, developers can streamline their development processes, ensure consistent build and test environments, automate tedious tasks, and foster collaboration among team members. The flexibility and extensibility of GitHub Actions make it a valuable tool for modern software development and continuous integration/continuous deployment (CI/CD) pipelines.
Let’s get started in devising a workflow to fetch multiple secrets from Hashicorp Vault.
First we need to place the YAML file in the .github
folder, follow these steps:
- Create a folder named
.github
in the root directory of your project if it doesn't already exist. - Inside the
.github
folder, create another folder namedworkflows
. - Move your YAML file into the
workflows
folder.
The file structure should look like this:
- .github
- workflows
- your-yaml-file.yml
Final YAML file to install all the required dependencies using the Dockerfile and fetching the secure secrets from vault.
name: GHA Workflow
on:
push:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v2
- name: Set up Docker
uses: docker/<reference your dockerfile repo name here>@v1
- name: Log into HashiCorp Vault
uses: hashicorp/vault-action@v2
with:
url: ${{ secrets.VAULT_URL }}
token: ${{ secrets.VAULT_TOKEN }}
- name: Fetch Secrets from Hashicorp Vault and declaring it as env variables
run: |
data_1=$(vault kv get -field=data_1 secret/data/myapp)
data_2=$(vault kv get -field=data_2 secret/data/myapp)
data_3=$(vault kv get -field=data_3 secret/data/myapp)
secret_1=$(vault kv get -field=secret_1 secret/data/myapp)
secret_2=$(vault kv get -field=secret_1 secret/data/myapp)
secret_3=$(vault kv get -field=secret_2 secret/data/myapp)
echo "DATA_1=$data_1" >> $GITHUB_ENV
echo "DATA_2=$data_2" >> $GITHUB_ENV
echo "DATA_3=$data_3" >> $GITHUB_ENV
echo "SECRET_1=$secret_1" >> $GITHUB_ENV
echo "SECRET_2=$secret_2" >> $GITHUB_ENV
echo "SECRET_3=$secret_3" >> $GITHUB_ENV
- name: Printing the values
run: |
echo "The Value of DATA_1 is:" ${{ env.DATA_1 }}
echo "The Value of DATA_2 is:" ${{ env.DATA_2 }}
echo "The Value of DATA_3 is:" ${{ env.DATA_3 }}
- name: Deploy to Azure
shell: pwsh
working-directory: Azure/scripts/
run: |
./Deploy.ps1 -ID ${{ env.DATA_1 }} -TeamName ${{ env.DATA_2 }} -Workspace ${{ env.DATA_3 }}
- name: Validation
run: |
# Add commands to validate
- name: Run Tests
run: |
# Add commands to run tests against the deployed Azure resources
Creating a Docker container action reference link :
https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action
Please find the Dockerfile below to install Powershell and Azure module to execute the commands.
# Use an official PowerShell runtime as the base image
FROM mcr.microsoft.com/powershell:latest
# Set the working directory inside the container
WORKDIR /app
# Install Azure PowerShell module
RUN pwsh -Command "Install-Module -Name Az -Force -AllowClobber"
# Copy the application source code to the working directory
COPY . .
# Specify the command to run the application
CMD ["pwsh", "-File", "app.ps1"]
You can customize this Dockerfile based on the specific dependencies and requirements of your PowerShell and Azure application. Once you have the Dockerfile ready, you can build an image using the docker build
command and then run a container based on that image using the docker run
command.
The below step is to fetch secrets from the Hashicorp secret and storing it in a variable. The variables will be declared as an environment variables and we can leverage those in the other step.
- name: Fetch Secrets from Hashicorp Vault and declaring it as env variables
run: |
data_1=$(vault kv get -field=data_1 secret/data/myapp)
data_2=$(vault kv get -field=data_2 secret/data/myapp)
data_3=$(vault kv get -field=data_3 secret/data/myapp)
secret_1=$(vault kv get -field=secret_1 secret/data/myapp)
secret_2=$(vault kv get -field=secret_1 secret/data/myapp)
secret_3=$(vault kv get -field=secret_2 secret/data/myapp)
echo "DATA_1=$data_1" >> $GITHUB_ENV
echo "DATA_2=$data_2" >> $GITHUB_ENV
echo "DATA_3=$data_3" >> $GITHUB_ENV
echo "SECRET_1=$secret_1" >> $GITHUB_ENV
echo "SECRET_2=$secret_2" >> $GITHUB_ENV
echo "SECRET_3=$secret_3" >> $GITHUB_ENV
Integration of GitHub Actions, Dockerfile, and HashiCorp Vault provides a powerful solution for streamlining software development workflows. By leveraging GitHub Actions, developers can automate various tasks, from code deployment to testing, while Dockerfile integration simplifies the installation of dependencies and ensures consistent environments. Furthermore, the integration of HashiCorp Vault enables secure retrieval of secrets. This combination of technologies enables efficient, secure, and automated software development processes, enhancing collaboration, accelerating delivery, and fostering a continuous improvement mindset. By embracing these tools, developers can optimize their workflows, ensure consistent and reliable builds, and enhance overall productivity in their software development projects.
Thanks for your time. Happy Learning !
Follow me on LinkedIn:
https://www.linkedin.com/in/tejas-c-s-439a021b1/