Before you do infrastructure as code, consider your workflow carefully
What happens with infrastructure as code when you want to make a change on prod?
I’ve been working on automation for a few years now. When you build your cloud infrastructure with code, you really take everything to a whole new level.
Join 38,000 others and follow Sean Hull on twitter @hullsean.
You might be wondering, what’s the day to day workflow look like? It’s not terribly different from regular software development. You create a branch, write some code, test it, and commit it. But there are some differences.
1. Make a change on production
In this scenario, I have development branch reference the repo directly on my laptop. There is a module, but it references locally like this:
source = “../”
So here are the steps:
1. Make your change to terraform in main repo
This happens in the root source directory. Make your changes to .tf files, and save them.
2. Apply change on dev environment
You haven’t committed any changes to the git repo yet. You want to test them. Make sure there are no syntax errors, and they actually build the cloud resources you expect.
$ terraform plan
$ terraform apply
fix errors, etc.
3. Redeploy containers
If you’re using ECS, your code above may have changed a task definition, or other resources. You may need to update the service. This will force the containers to redeploy fresh with any updates from ECR etc.
4. Eyeball test
You’ll need to ssh to the ecs-host and attach to containers. There you can review env, or verify that docker ps shows your new containers are running.
5. commit changes to version control
Ok, now you’re happy with the changes on dev. Things seem to work, what next? You’ll want to commit your changes to the git repo:
$ git commit -am “added some variables to the application task definition”
6. Now tag your code — we’ll use v1.5
$ git tag -a v1.5 -m “added variables to app task definition”
$ git push origin v1.5
Be sure to push the tag (step 2 above)
7. Update stage terraform module to use v1.5
In your stage main.tf where your stage module definition is, change the source line:
source = “git::https://github.com/hullsean/infra-repo.git?ref=v1.5"
8. Apply changes to stage
$ terraform init
$ terraform plan
$ terraform apply
Note that you have to do terraform init this time. That’s because you are using a new version of your code. So terraform has to go and fetch the whole thing, and cache it in .terraform directory.
9. apply change on prod
Redo steps 7 & 8 for your prod-module main.tf.
Related: When you have to take the fall
2. Pros of infrastructure code
o very professional pipeline
o pipeline can be further automated with tests
o very safe changes on prod
o infra changes managed carefully in version control
o you can back out changes, or see how you got here
o you can audit what has happened historically
Related: When clients don’t pay
3. Cons of over automating
o no easy way to sidestep
o manual changes will break everything
o you have to have a strong knowledge of Terraform
o you need a strong in-depth knowledge of AWS
o the whole team has to be on-board with automation
o you can’t just go in and tweak things
Related: Why i ask for a deposit
Get more. Grab our exclusive monthly Scalable Startups. We share tips and special content. Our latest Why I don’t work with recruiters
Originally published at Scalable Startups.
