Github Organization as a Code
Github became de-facto standard code storage platform for all types of organizations, from small startups to huge corporations. While a company evolves and grows it will face the need for keeping code organized, structured and secured. Of course, it can be done by a sysadmin person while you have one team and couple members. But what about 5 teams, 10 teams? At some point, it will become extremely hard to keep track on naming convention and access permissions for many users, teams, and repositories.
Luckily most of this problems can be solved by infrastructure management tool that has been around for a while: Terraform. Terraform provides an ability to manage infrastructure as a code on different platforms like AWS, Azure, Kubernetes and also Github.
From Day0
The easiest way to integrate Terraform is to start using it from day 0. While is not always possible, I will cover a topic of existing organization migration later. For now let’s create an organization with a couple teams, repositories, and projects. I will assume that you’re familiar with Terraform basics. To learn about Terraform, follow the official Getting Started guide.
Due to provider limitation, we can’t create organization using Terraform. For organization creation follow this guide. To generate a new personal access token, check this step-by-step tutorial. I recommend using the permission setup shown below.
Assuming we have organization and token created, we can continue to the provider configuration. Github provider allows to configure multiple parameters, such as token, organization name, organization base_url and insecure flag to skip SSL verification. For demo purposes, we only specify organization and token parameters linked to input variables.
Now we can create a simple organization configuration using available resource. Our demo organization will consist of a team with members and a repository with admin access for this team. Please use your own Github username for this demo.
To check and apply our configuration changes, we need to execute following terraform commands.
Now we can preview our changes before applying it to the infrastructure.
Now we can take a look at infrastructure changes that we are going to apply in the next step. Since we don’t have anything pre-configured, all 4 resources are going to be created.
This step plays a crucial part in the deployment process since we can catch any inconsistencies or unnecessary changes before they will be pushed to an actual organization.
After validating that everything looks like we expected, we can move forward and deploy the changes to the organization. We’ll be required to provide github_organization and github_token variables manually.
Otherwise, we can place these variables into testing.auto.tfvar so they will be loaded automatically.
Be careful and never commit your token into any public repository.
Now we can go to the organization and see changes above applied:
The organization with one team, one repository, and one member
Team demo-team-1 with a user attached
Repository demo-repository with admin access for demo-team-1.
After making some changes, for example, downgrading demo-team-1 permissions to pull in demo-repository, we repeat plan and apply operations to update infrastructure.
Now new permission setting was applied to the repository.
As we can see Terraform provides a convenient way to store the whole Github organization configuration as a code. We can also version it and automate changes review and deployment with Jenkins and Github pull request hooks.
From Day1000
It’s easy to commit building infrastructure-as-a-code from scratch, but can we move existing Github organization to Terraform resources? The answer is yes!
To import existing resource into Terraform first we need to create appropriate resource declaration in our configuration.
Let’s add test-import github_repository resource into the configuration, but leave all fields blank. We need to create
The next step is to import resource to the Terraform state with import command, where the first argument is a name of the resource and the second one is repository name in the organization. After the import is done, we can check resource configuration from the state.
Now we can move this configuration parameters to the configuration. The trick is to remove computed values from this configuration. Using github_repository resource documentation and terraform plan command, make sure that configuration copied correctly and won't cause any changes in the organization. The end result should be like this:
Be VERY cautions of some parameters, like auto_init for github_repository since it can cause full repository recreation and losing your data in this repo. To avoid this issue always check your changes before applying configuration changes, repositories backend are highly recommended as well.
Bonus: Repositories Backup
Here is a little bonus — a very useful article how to setup automated Git repositories backup with AWS S3 and Lambda.
Git Webhooks with AWS services
Conclusion
No matter how many users, repositories and teams are in an organization. Terraform provides an easy way to define, organize and version all kind of resources and permissions for Github, as well as recreate organization structure from scratch any time.
All operations from creating new repositories to managing access rights can be automated with automated Jenkins pipeline and Github Pull Request, so there won’t be any need to do any of this manually anymore.
