Previously we covered high level Pulumi concepts and compared it to Terraform. Today let’s dwell a bit more on the available Pulumi features like default service backend, state and secrets management or available Terraform integrations.
Default service backend
Prior to creating your first project, by default, you have to create an account on https://app.pulumi.com and login from your CLI using
pulumi login command.
CLI is tightly coupled to the default service backend. Once you are logged in, each project you create will be showed in the web dashboard. The checkpoint files (infrastructure state) will be also kept there. That’s a bold decision taking into consideration that checkpoints contains secrets and passwords to your cloud services.
Default service backend basic usage is free of charge, however, its offer is tiered, and depending on features you want, you might need to upgrade to paid version.
In the free Community Edition version dashboard you can review the projects, check the activity log and list the available resources in your stacks grouped by categories like Containers, VMs, SQL, Queues and etc.
Notable features in paid versions are:
- CI/CD integration
- Basic or advanced secrets management
- Self hosting the https://app.pulumi.com
- Automatic state locking
- Deployment history (supports rolling back to previous infrastructure state)
Available tiers and features can be checked here.
Pulumi just like any other IaC tool has to keep the current state of the infrastructure somewhere. Pulumi stores it in so called checkpoint files.
As described above, by default, checkpoint files are stored in the default service backend. For security reasons this will be a big no for many companies. These files may contain passwords and keys, thus, it is not recommended to host them at a third party SaaS provider.
However, Pulumi supports third party backends like S3 or GCS. You can store checkpoints, for example, in your’s company AWS account where you are the one controlling access policies.
Switching state backend to S3 bucket is very easy and will be presented in next installment. Unfortunately, as of today, self-managed backends do not support state locking. If you do not plan to buy paid Pulumi subscription that includes locking, you will have to implement it by yourself.
It shouldn’t be a big problem — your language of choice probably already has a decent locking library, e.g., Python’s python-dynamodb-lock.
Each stack can be configured separately. By default, when you issue
pulumi stack create command a new stack configuration file will be created. It will be named Pulumi.[stackname].yml. Primary reasons for existence of this file are:
- Storing the secrets provider configuration (explained in next section),
- Storing stack variables
To store an variable in configuration file you should use
pulumi config setcommand. Note that variable is also prefixed by a stack name.
$ pulumi config set examplevar "examplevalue"
$ grep examplevar Pulumi.dev.yaml
How to reference these variables in program depends on language you use. Python example:
cfg = pulumi.Config()
val = cfg.require('examplevar')
val2 = cfg.get('examplevar')
Difference between cfg.require() and cfg.get() is that in case of absence of the variable the former will raise an exception and the latter will return none.
By default, pulumi CLI uses per-stack encryption key and salt to encrypt the secrets. Encryption is done at https://app.pulumi.com, not locally. To add an encrypted variable to stack config you have to provide
--secret flag to the
pulumi config set command.
pulumi config set --secret influxdb_admin_password S3CR3T_PA55W0RD
Encrypted secret will be stored alongside stack variables. This is an imperfect solution — you end up with secret hashes in VCS —they are hashes, but still, having them in VCS is considered a bad practice.
However, Pulumi uses general purpose languages to manage infrastructure; you can easily obtain your passwords from AWS SSM or Azure Vault and avoid using the built-in secrets management function. You can even use native Pulumi primitives for that:
somepassword = pulumi_aws.ssm.get_parameter(with_decryption=True, name="/some/password_in_ssm")
As said before, default encryption provider is https://app.pulumi.com. You can change it to an alternate provider. My personal opinion is that it is wiser to choose external encryption provider like, e.g., KMS in your AWS account. Not only you control the key revocation process, but you also have access to whitepapers describing how encryption is done, and how high entropy is achieved. Currently supported encryption providers are:
- AWS KMS
- HashiCorp Vault
- Azure Vault
- GCP KMS
Encryption provider cannot be changed for already created stack(or at least there is no documentation on that). You must specify encryption provider when creating a stack:
pulumi new aws-python --dir mon --secrets-provider="awskms://KMS_KEY_ID?region=eu-west-1"
Pulumi developers created an terraform-bridge which is a library that allows to statically compile Terraform providers to use with Pulumi. In fact that’s how many of the Pulumi providers were created (or ported)— it is an interesting fact, however, unless you want to bring new Terraform provider to Pulumi, you don’t need to know how it works.
To support soft transition from Terraform to Pulumi a pulumi-terraform package was created(in description it is also referred to as bridge so it is a bit confusing). You can use it to reference infrastructure created by Terraform in Pulumi programs. For example, using Pulumi you may deploy an application in a VPC referenced from a Terraform state.
In next installment we will use the knowledge acquired so far to create an real infrastructure using Pulumi! Stay tuned.
Feel free to contact me at firstname.lastname@example.org. If you want to receive my posts over your email subscribe to my substack at https://cloudtalks.substack.com/.