Part 1 — HumanGov Application — Git & GitHub-1: Introduction to Working with Git and GitHub Repositories in AWS Cloud9 Environment
HumanGov is a software-as-a-service (SaaS) cloud company that will create a Human Resources Management SaaS application for the Department of Education across all 50 states in the US and host the application files and databases in the cloud. Whenever a new employee is hired, a new registry will be created for this employee inside this application.
In this following project series, we are going to transition the architecture from a traditional virtual machine architecture to a modern container-based architecture using Docker containers and Kubernetes running on AWS. In addition, we will also be responsible for automating the complete software delivery process using Pipelines CI/CD using AWS services such as AWS CodeCommit, AWS CodePipeline, AWS CodeBuild, and AWS CodeDeploy. Finally, we will learn how to monitor and observe the cloud environment in real-time using tools such as Prometheus, Grafana, and automate one-off cloud tasks using Python and the AWS SDK.
In this section, we are going to introduce AWS Cloud9 Environment with Git and GitHub and practice before using it in the implementation of the HumanGov application. This is the 1st part of a project series.
As a DevOps Engineer for a Cloud Software as a Service Company called HumanGov that is responsible for provisioning, deploying, and maintaining a Human Resources Management Cloud application using DevOps practices for the Department of Education across multiple states in the USA. I was responsible for provisioning, maintaining, and monitoring the AWS cloud infrastructure using Infrastructure as Code and Configuration Management automation tools such as Terraform and Ansible while keeping the configuration files centralized in a Git repository.
This project consists of 2 parts.
Part 1: Introduction to Working with Git and GitHub Repositories in AWS Cloud9 Environment
Part 2: HumanGov: Implementation of Git Repositories for Application and Infrastructure Code Using AWS CodeCommit and Proof of Concept (PoC) Process of Commit, Push, and Revert Code Changes
We are going to use a cloud-based working environment instead of connecting and working on a remote environment on AWS from your laptop. Therefore, you will not need to install necessary tools such as Terraform, Docker, Kubernetes, Git, etc. on your local computer. All these tools are already available to be used in a cloud-native environment. First, we need to create an environment on AWS to access these tools to execute tasks.
Step 1: Create an AWS Cloud9 Environment with Linux Operating System
We are going to access the AWS cloud environment through AWS Cloud9 Console which is a web-based console similar to Visual Studio Code and allows you to have an EC2 instance (virtual machine) working environment on AWS. All your tasks and tools will be executed in this instance.
Name: humangov
AWS Cloud9 connects to your EC2 instance through the AWS Systems Manager. It opens internet connectivity to the EC2 instance without opening an SSH port to the internet.
Step 2: Connect to the AWS Cloud9 IDE
Connect AWS accounts first and connect AWS Cloud9 Console.
Once you are connected to Cloud9 Console, it opens an encrypted SSH connection to the EC2 instance and web-based IDE to execute our tasks as a DevOps Engineer so you can start creating configuration files for Terraform, Ansible, Docker, Kubernetes, etc., and executing commands in the terminal without installing any tool.
You will be working with Software Developers who are responsible for developing the HumanGov application. Every software developing company needs a Version Control System like Git in our case where the developers can store the source code of the software application in a safe way and collaborate with other developers. As a DevOps Engineer, you are responsible for creating Git repositories and storing configuration files in them to deploy and configure infrastructure. All files will be accessible to everyone working on the project.
Step 3: Distributed Version-Control System: Git on a local computer
Git is suitable for storing software source codes and their versions in repositories. It works only locally on your computer. GitHub is a repository hosting service allowing you to transfer your local git repository to the cloud and access your source code remotely. Other team members also can access and download it to work collaboratively and make changes to it. Git is a software, and GitHub is a service. They both provide Source Code Management (SCM) and allow tracking changes from previous, current, and new versions with the ability to roll back in case of bugs and mistakes. You can document history; who made it, when, and what changes have been done.
We are going to use a cloud-based environment that git is already installed but you can download Git to your local computer by following the link below:
After downloading, you can create a new working directory and a git repository inside this new directory.
mkdir local-repo
cd local-repo
git init
git init creates a hidden folder .git. Git uses this folder to store its files and control versions in it. Once the git init command runs you create a new git repository with a new branch.
You can download a git cheat sheet from the link below:
There are 3 areas where files/folders/changes can be located:
- Working Directory: The root directory that the git repository will be created in it.
- Git Repository: Hidden .git folder inside the working directory.
- Staging Area: Between the working directory and the git repository. The area file changes move from the working directory to the git repository. You should first move changed files from the working directory to the staging area before moving to the git repository using the git add command. Then you can run the git commit command to move files from the staging area to the git repository (.git/object/) to be able to track changes. You can use the git checkout command to go to certain previous versions and the git status command to see files in the staging area.
Every commit process has an author name, author email, and commit description. Everybody can see who committed and what has been committed. Once you commit the changes, the commit will include your name and email address. Setting up the author's name and email:
git config --global user.name <your user name>
git config --global user.email <your email>
# check user name and email for confirmation
git config --list
Before committing the changes the git repository only had info and pack folders. After committing, 3 files named 20, 80, and fd were added to reference changes. Further, you can use the git log command that shows the author name, email, commit date, and commit description.
Step 4: Git on AWS Cloud9 Console
Go to the Cloud9 Console and check if the Git is already there. It should be pre-installed.
git --versionThen follow all the steps you did on the local computer from Step 2 except the installation part.
Commit changes to the git repository. Do not forget to set up your username and email address before committing.
Step 5: Working with Git Object Types on AWS Cloud9 Console
Git has 4 types of objects:
- Blob: Represents files inside a git repository
- Tree: Represents folders/directories
- Commit: Points to a single tree, a snapshot of a file in a given time.
- Tag: Marks a specific commit
Every git object in the Git Repository has a 40-character long hash which is a unique identifier created using a specific hash function SHA1. GitHub shows only the first 7 characters of the SHA1. The first 2 characters of SHA1 are used while storing objects under .git/objects/ folder.
# get SHA1 hash using folder name with first 2 character
# This command gives 38 characters excluding first and second characters(80 in this case) of SHA1
ls .git/objects/80
# get the object type using SHA1 hash
git cat-file -t 80849fb1f3ef4f14d9d421c67f964434f57069dc
# get content of the object using SHA1 hash
git cat-file -p 80849fb1f3ef4f14d9d421c67f964434f57069dc
Run some of these commands on the git repository in your AWS Cloud9 Environment. Add 2 characters from the folder to the beginning of the hash you got while running the git cat-file -t and git cat-file -p commands.
Step 6: Merging
Once new features are needed to be added new branches should be created. After the development process is completed on the new feature, the new branch should be merged with the main branch and then it will be deleted.
# Check current branches
git branch
# Create a new branch feature1, all files from main branch will be automaticaly included to this new branch
git branch feature1
# Switch to feature1 branch
git checkout feature1
# All changes after this point will be done on feature1 branch, nothing will change on main branch.
# Switch back to the main branch and merge it with feature1 branch to apply changes
# You need to be in the branch you want to merge into.
git checkout main
git merge feature1
# Delete/trim the feature1 branch from the tree
git branch -d feature1
Step 7: Working with Remote Repository GitHub
What we have done so far (on our local laptop and Cloud9) is on our local repository. All source code should be located in a safe remote repository that everyone in the team can access and work on it collaboratively. This remote repository is actually a server inside the cloud. We will have a copy of our local repository inside that remote repository. The remote repository has a specific name: origin.
To copy and synchronize the files in your local repository to the remote repository we execute the push command. Someone else can execute the pull command to download source files from the remote repository to the local repository to make changes to it and push them back to the remote repository with its new feature/version.
As a DevOps Engineer, your responsibilities are to help the Software Developers with accessing the remote repository without any problem and maintaining this repository. You will also have your own repository to store infrastructure as code configuration files like Docker, Kubernetes, Terraform, etc.
GitHub is one of the widely used remote repository services. In addition to GitaHub, cloud providers such as AWS, Microsoft Azure, and GCP also have their managed Git repositories.
AWS > AWS CodeCommit
Microsoft Azure > Azure Repos
GCP > Cloud Source Repositories
First, create a new account on GitHub if you do not have one. We can upload our source files from our Cloud9 local repository to this GitHub repository.
Create a private remote repository on GitHub that we can synchronize with the Cloud9 repository later on.
You can also run git commands to create a local repository (according to GitHub Quickstart)
Go to Cloud9 Console. Create a new folder.
Right-click on environment name humangov > new folder > name the folder > go inside this folder > initialize a new git repository in this folder
Add a remote repository. We are going to add a remote Git configuration so that the local Cloud9 repository can know the remote destination where the source files will be uploaded. After configuration, push the local repository to Github main branch.
# Connect to the GitHub repository
git remote add origin https://github.com/ctekin22/simple-website-repo-poc1.gitGo to the Cloud9 Console. Create a new file inside the folder (in my case local-repo2) you created previously. Create a new file (index.html in my case) you wish inside it by right-clicking. Add this file to the local git repository first.
Now, we can push the changes we made to the remote git repository on GitHub. It is going to ask for a username and password. username is the account name you used while creating your account. For the password section, GitHub does not allow you to use your account password to log in from the terminal for security reasons. You need to generate a token on your GitHub account to access your account from Cloud9.
GitHub profile > Settings > Developer settings > Personal access tokens > Tokens(classic) > Generate new token > Generate new token (classic) > Select the scope repo to give full access to the private repository > Generate token
Copy the generated token and paste Cloud9 to complete the push process.
All changes are now pushed to the GitHub repository. Go to the GitHub repository and check if the repository is updated.
All files from the local Cloud9 repository are in the GitHub repository now.
Now let's do some updates on the index.html file inside Cloud9 local repository.
This is our current file preview on Cloud9.
Change: Welcome to My Website >> Welcome to HumanGov
When you run git status you can see which file you modified. Commit changes to the git repository in Cloud9.
The file in the GitHub repository still has the old version. Push the changes to the remote GitHub repository as we did before to update files with the new version.
Go to the GitHub repository and you can see updated files with the given commit messages.
CONGRATULATIONS!!
Follow me on LinkedIn:
