Trading stocks and Terraform
I’m John Renne, the DevOps lead within BUX. Having been in the field of development and infrastructure since 1998 I’m still passionate about new technology. I’m a strong infrastructure as Code advocate and dedicated to automate as much as possible. One of the fun parts op my job, is I sometimes get the chance to give people the opportunity to take a little peak behind the curtains. Sometimes during a meet-up, sometimes through a blog. Today I decided it was time to share with you why we are strong believers in Infrastructure as Code and how we experienced our journey into it.
How it all started.
I think most people working in DevOps know the drill. Infrastructure isn’t clicked together in the console of your favourite cloud provider, it’s set up using infrastructure as code. In the case of BUX we decided to use terraform to provision our infrastructure. We can easily create and delete resources, and everyone’s happy. As Shakespear would say it “All’s well that ends well” but this wasn’t the end of it…
The next nuisance.
Working in a tech company with a reasonably modern stack probably means you’re familiar with microservices and one of the results is a heap of git repositories. Every time a new service is created, a new git repository is created too. That’s where the github provider showed up. I mean, every developer knows git, so why not make sure they can easily create new git repositories with a pull request? Using the terraform to manage our GitHub repositories, we accomplished exactly that.
When teamleads are techies.
After the repositories it was time for the next step. I mean, we had the repositories, and the infrastructure, but how do you keep track of who exactly should have access to infrastructure and repositories? Again terraform helped us out. All of our teamleads are techies, which means they know git. Why not also put access management in git and automate it in exactly the same way as we did with our infrastructure and repositories. With the help of (again) the GitHub provider and the google (workspace) provider we again managed to enable people and made sure the proces followed what we know, development best practices with a PR and peer reviews.
What about a disaster recovery?
With all of this out of the way we focussed on the next part. We had terraformed the infrastructure and the repositories, but we weren’t done yet. The next part was disaster recovery. We mainly used helm to manage our deployments and knowing there is a helm provider for terraform, made it an obvious choice. We were gonna terraform the services, so when the infrastructure was rebuilt, automatically all services would get deployed, without having to run all CI/CD pipeline associated with them. The idea was simple, but the implementation was a bit trickier.
We could have implemented terraform in all CI/CD pipelines we used, but that didn’t make sense to us. Every team is maintaining their own CI/CD and not all developers are fluent in terraform. We had to find a trick, and we succeeded. We included all helm releases in our infrastructure pipelines. That way we didn’t only manage the infrastructure but also the services running on top of it. To make sure we kept in sync with the services, each service commits the latest release to the infrastructure repository for that specific environment. That way all we need to do to rebuild a complete environment with all required services, is run the infrastructure pipelines and we’re done.
What the future will bring.
Are we done yet? Nope not by a long shot. There’s so much more we want to do and that can be done. We have some really nice items on our backlog that can also be automated, like slack, pagerduty, etc. Are we moving in the right direction? Who will tell? The recent change in license for terraform was a surprise, but didn’t bite us (yet). Do we think we took the right approach? 100% By automating and adapting our processes so they follow standard software development processes, we made sure they’re done in a way we all know and we’ve all worked with for years. It made life easier for a lot of people.
Want to know how we’re progressing? Just keep an eye on this page, and get a nice sneak peak Inside BUX.
