Backstage and Terraform — A Powerful Combination for Ops, Wonderful for Devs
Even as the DevOps/Platform world is in constant evolution, classical challenges persist in many companies, especially those with extensive adoption of cloud-native technologies.
Certainly, you must have come across the famous scenario of Infrastructure as Code (IaC).
Using IaC to manage cloud resources through code has become virtually indispensable today.
Although this approach has been proving itself over time, as adoption increases, it’s natural for some challenges to begin to affect the day-to-day of teams. Perhaps you, as a part of a development team, can relate to the scenario.
When needing to create a new resource in your cloud, you realize that it will be necessary to open a ticket for the DevOps/SRE team to create the resource for you. This is because all resources are created only with Infrastructure as Code (IaC)).
And this creation takes a considerable amount of time from the development time of your functionality. Or perhaps you even have the autonomy to create the Pull Request for the resource creation yourself, but you find yourself having to learn a new language (Terraform) to create your resource.
Could it be simpler? And maintain the benefits for both the usability of development teams and platform teams? I have good news! 😄
By now, you’ve probably come across the term Platform Engineering and likely encountered the concept of a Developer Portal.
You might have also heard of Backstage.io.
(I have another post to introduce you to it if you haven’t already - LINK).
I want to introduce you to a feature called Software Template that helped me with the problem I mentioned at the beginning of the post, and I believe it can help you too.
Imagine that you can create a form with various fields and run a pipeline that generates Terraform code for you.
For developers, it’s just a page with some fields.
For maintainers, it’s a YAML file and a template mechanism quite similar to Jinja/Helm that helps generate the files you need, opening a pull request with all the information you want.
So, how do you use it?
The entire secret lies in a single file, where we will configure both the form and the execution pipeline of the actions, following a format quite similar to a Kubernetes Custom Resource Definition (CRD), with kind, metadata, and spec.
This file is separeted into 4 parts.
metadata
These are the pieces of information used to display your template on the main page of Backstage, such as title, description, responsible team, and we can add documentation links and some tags that can be used to categorize the templates.
parameters
The parameters are where we declare how the data input form will be. Behind the scenes, this form uses react-jsonschema-form, so we can create fields from simple text to a dropdown with various options. It’s also possible to customize other aspects, such as whether the text is sensitive information, and even add validations using regex.
This is extremely useful because it allows us to control very well the possibilities and patterns used in the values that will be chosen by the user.
The Backstage documentation itself has some examples of how we can use it.
A good tip is to use the Template Editor, which gives you quick feedback on how your form will be presented.
We can also create custom inputs to obtain external information for Backstage, such as making a request to another API and displaying it in the dropdown.
In the Backstage documentation, we have more information:
But we can create another post showing in practice how to do it 😄
actions
The actions are where platform teams can show their best 👩🏽💻✨👨🏽💻.
Think of it as creating a pipeline.
We can create multiple steps, using native Backstage actions or some that can be installed as plugins.
Each step has the following fields:
- id: A unique identifier that we use to reference in other parts of the file.
- name: Text that will be displayed in the UI during the execution of steps.
- action: The identifier of which action you are using.
- input: These are the variables that we can pass as arguments to the action.
If you’ve used GitHub Actions, you’ll feel familiar.
To check all the actions your Backstage has, there’s a /create/actions page where you can see the name of the action and the inputs it can receive.
And in these actions, we can use the values entered in the form as inputs using the syntax ${{ parameters.field-id }}.
Example:
The outputs are links that we can display at the end of the execution of steps.
This way, we can already provide some links for the user to follow the next steps or see the result of what was created.
A good example is to add the link to the Pull Request that was created.
Now, combining everything, let’s look at a practical example.
Considering we already have an infrastructure monorepo using Atlantis and Terraform, let’s automate the creation of GitHub Pull Requests for creating SQS queues.
This allows any team to create pull requests autonomously, following a pattern created by the DevOps/SRE team.
The file would look like this:
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
name: sqs-with-dlq
title: Standard AWS SQS
description: PR to create AWS SQS Queue
tags:
- infrastructure
- terraform
- aws
- queue
links:
- title: Documentation
url: https://backstage.io/docs/features/software-templates
icon: docs
- title: Source
url: https://github.com.br/gabriel-dantas98/backstage-scaffolders/blob/main/basic-react-app/create-web-app.yaml
icon: github
spec:
owner: cloud_platform
type: infra-a-code
parameters:
- title: Inform the SQS initial information
required:
- queue_name
properties:
queue_name:
title: Queue name
type: string
description: |
The name of the queue
environment:
title: The environment the resource is part of
type: string
description: Which environment your application will be created in is usually related to the AWS account you will be provisioning your resource to.
default: prod
enum: [prod, staging, dev]
create_dlq:
title: Create a DLQ with redrive policy?
type: boolean
description: If you desire to create a DLQ with redrive policy
default: true
ui:widget: radio
- title: Information about ownership (using in tags)
properties:
tribe:
title: Tribe
type: string
description: |
Select the Tribe owner of resource, we used as a tags in AWS
ui:field: OwnerPicker
ui:options:
allowedKinds:
- group
squad:
title: Squad
type: string
description: |
Select the Tribe owner of resource, we used as a tags in AWS
ui:field: OwnerPicker
ui:options:
allowedKinds:
- group
steps:
- id: template
name: Render terraform files
action: fetch:template
input:
targetPath: ./templates/outputs
url: ./skeleton
values:
queue_name: ${{ parameters.queue_name }}
create_dlq: ${{ parameters.create_dlq }}
environment: ${{ parameters.environment }}
squad: ${{ parameters.squad }}
tribe: ${{ parameters.tribe }}
- id: show_workspace
name: Show workspace files
action: debug:log
input:
listWorkspace: true
- id: terraform_pr
name: Create terraform PR
action: publish:github:pull-request
input:
repoUrl: github.com?owner=gabriel-dantas98&repo=piltover-infrastructure
branchName: 'sw-template/sqs/${{ parameters.queue_name }}'
title: '🔩 Create ${{ parameters.queue_name }} AWS SQS'
description: |
## Creating SQS ${{ parameters.queue_name }}
This is an initial pull request to create an SQS queue and was created based on the Backstage template.
If you need to add more parameters, check the official documentation - https://registry.terraform.io/modules/terraform-aws-modules/sqs/aws/latest
*created by: [Backstage Software Template](https://hextech-portal.gdantas.com.br/create)* 👷♂️⚙️👷♀️
sourcePath: ./templates/outputs
targetPath: 'aws/production/sqs/${{ parameters.queue_name }}'
- id: label_pr
name: Add labels to PR
action: github:issues:label
input:
repoUrl: github.com?owner=gabriel-dantas98&repo=piltover-infrastructure
number: '${{ steps.terraform_pr.output.pullRequestNumber }}'
labels:
- terraform
- created-by-backstage
- ${{ parameters.environment }}
- sqs
output:
links:
- title: 'Go to pull request :D'
url: ${{ steps.terraform_pr .output.remoteUrl }}
icon: github
- title: 'To view more check documentation'
icon: docs
url: "https://registry.terraform.io/modules/terraform-aws-modules/sqs/aws/latest"
After executing, you will have the following Pull Request:
With this, having Atlantis applied to your team’s repositories, you just need to go through a review and apply the queue creation.
For the post not to become more extensive, I’ve separated some links that can help with the Atlantis configuration:
A nice blog on how to install and use Atlantis: https://dev.to/brunopadz/iniciando-com-atlantis-5dcd — Bruno Padilha
If you prefer video, this playlist will surely help: https://www.youtube.com/watch?v=r-89xbvR004&list=PL9GsBqyuVC-zAUMBELzLZEoe9tiOsEM5Q — Bruno Russi Lautenschlager
Tips I wish I had received and want to share
- Try to be as simplistic as possible; think that this form can be used by someone with little or much time within the company.
- Use descriptions liberally; it’s the best tool for communicating and providing context for those filling out the form.
- Use architecture and flow images in Markdown where possible; this instills more confidence and visibility. Take the opportunity to insert links to documentation that can assist those using the template.
Don’t get attached to the tool, get attached to the solution ;)
If you have any suggestions for a future post, suggestions are welcome! 🚀
Thank you for making it this far! Before you go:
- Don’t forget to 👏 and follow!
- More cool stuff at gdantas.com.br 🚀
- If you want to stay stylish, check out our t-shirts at https://reserva.ink/deployou 👕💻
