Setting up your GCP foundations through Terraform — Chapter 3 — Deploying the bootstrap and CI/CD Projects
Introduction:
In this chapter, we will deploy through Terraform the seed and CI/CD projects. These 2 projects are the foundation that allows us to maintain terraform state in the cloud and to begin working in a GitOps model to deploy the rest of our environment through Terraform.
Finally, after walking through the implementation of the GCP module I will discuss some improvements which in my opinion we need to implement, and those will be the topics of the next articles in the series.
If you are enjoying this series and are excited to continue learning and building together, make sure to follow and/or subscribe on Medium, LinkedIn & Twitter.
If you have any questions, comments or suggestions on how we can make the series better, feel free to leave a comment at the bottom of the article — I’m really looking to hearing from you!
Let’s get to work:
* This section is a walk-through of the instruction on the Github page, with some additional clarifications and screenshots.
- Make sure you have worked through the prerequisites
2. Login to the GCP Console with the user that has the required permissions -> make sure you are on your org node -> click the cloud shell icon to get access to your cloud shell environment
3. Once your cloud shell environment is up and running — click on “Open Editor”
4. Click on “Terminal” -> “New Terminal” to open a terminal in your dev environment
5. Clone the GCP-Terraform-Example-Foundation by typing in the terminal git clone https://github.com/terraform-google-modules/terraform-example-foundation.git. You should see a new folder appear in your navigation bar.
6. Click the drop-down arrow and navigate to terraform-example-foundation -> 0-bootstrap
7. Right click on terraform.example.tfvars -> click Rename -> teraform.tfvars
8. Click on the newly edited terraform.tfvars this will open the editing window on the right-hand side of the screen. Edit the parameters in this file to match your organization’s setup. org_id, billing_account, group_org_admins, group_billing_admins, default_region.
If you are building this our in an existing org and do not want to create this folder of resources at the root node level you can also uncomment line 32 where it says “parent_folder” and provide your parent_folder id
9. In the terminal pain make sure you are on the 0-bootstrap folder by typing cd terraform-example-foundation/0-bootstrap
10. Run terraform init
11. Run terraform plan
12. This will print out all the changes that will be applied to your environment. I have some concerns with the results of this command which we will discuss later on in the guide.
13. I will skip setting up the terraform-validator, as I don’t like running pre-GA code in my production environment and especially not on the folder and projects which will be the foundation for my entire org.
14. Run terraform apply
15. Run terraform output terraform_service_account to get the email address of the admin. You need this address in a later procedure.
16. Run terraform output gcs_bucket_tfstate to get your Google Cloud bucket name from Terraform’s state.
17. Copy the backend: cp backend.tf.example backend.tf
18. Update backend.tf with the name of your Cloud Storage bucket.
19. Re-run terraform init. When you’re prompted, agree to copy state to Cloud Storage.
20. Run terraform apply to verify that state is configured correctly. You should see no changes from the previous state.
What have we built:
Following this procedure, we have built out a folder with 2 projects in it.
- One of the projects is the seed project which contains the GCS bucket for maintaining our terraform state in the cloud, the service account and the enabled core APIs.
- A second project for our CI/CD pipeline where we have cloud build, our cloud source repositories and a GCS bucket used for cloud artifacts.
Recommended improvements:
While Google has done some amazing work here setting up this repo to help new customers onboard to GCP quicker I think there are things we can do to improve the overall process.
One point which I pointed out earlier in the post and stood out to me is the length of the output of the terraform plan command. When a terraform plan is hundreds of lines long, no one is able to actually read and understand what is being done to the environment, this leads to confusion and ultimately outages.
This being said, if you are facing a deadline and you need to get things done, I recommend continuing work with the GCP CFT, it is a great resource!
Based on the best practices on refactoring terraform (written by Ami Mahloof a former DoiT-International engineer), this is what I think we should do.
- Build out the repo itself to match the terraform best practices, splitting the folders between environments, modules and resources.
- Refactor the code to use the already existing GCP modules for folders, project-factory, GCS & service accounts
- Working with Terragrunt to keep the terraform code DRY
- Build out automated testing using Terratest
- Deploy the pieces of the architecture in steps so we can learn and understand what we are doing.
Coming Next:
- Building out our Bootstrap environment from scratch, starting with creating our folder and assigning the required permissions to the required groups on the folder.
Resources:
- GCP-Terraform-Example-Foundation
- Best practices on refactoring terraform
- GCP Terraform modules: folders, project-factory, GCS & service accounts
