Decoupling IaC with SSM Parameter Store
Have you ever heard about AWS Systems Manager Parameter Store? If not, you should really get yourself updated as it is one of the most problem-solving services from the Amazon’s cloud. Basically, it’s a key-value store for your app’s configuration. It also works great with KMS and IAM, so you can have your data secured in a proper manner. However, using it just for app configs seems pretty boring, let’s get it to the next level! ☁️
If it still does not ring the bell, you can check out our old article about how we use Parameter Store for provisioning environment variables in containerized workloads.
What’s good about Parameter Store is that it is supported by almost everything you can imagine when working with AWS. It’s easy to create entries and to read them while provisioning your infrastructure. One of my original problems was how to use Terraform and the Serverless Framework together in a one project. I wanted to create my VPC in Terraform and then let serverless framework know in which subnets and security groups it should deploy my functions. I don’t like hardcoding stuff in CI/CD pipelines, so creating subnets and then putting their ids in pipeline’s config is not my way, it has to be dynamic.
Multiple tools
What should cloud ninja do? Let terraform execute a script to setup a Serverless’ pipeline? Nah, unnecessary external dependencies. Maybe read terraform’s remote state? Looks nice, but in the end — too complicated. We need something simple and you already know the answer. Let’s use the Parameter Store to decouple communication between different tools.
In terraform file I can create resource type aws_ssm_parameter and put interpolated values based on my infrastructure. Then, in serverless configuration, I can read it by using $(ssm:…) variable. Things get a little bit trickier when you need to parse a list, e.g. subnet ids. For terraform part I use a simple join and then split it with Fn::Split — a CloudFormation intrinsic function. Isn’t that awesome? ❤️
Overcome CloudFormation limits
If it’s not in IaC, then it shouldn’t be in your AWS account. CloudFormation is an amazing tool that we use heavily in Nordcloud. I really like creating multiple stacks and exporting values from one layer to another, so I keep everything organized in separate files, e.g. VPCs, ECS cluster, Application stack.
Here comes the not-so-nice thing. If you export a value in one stack and then import it in another stack, you cannot update the first stack (if it requires a change in stack’s output). Use case? Let’s say we have stack A with SQS queue and its URL is exported. Then we create stack B which reads this value and sets it as Lambda’s environment variable.
Now we want to recreate this SQS queue, say, just because we don’t like the name we chose for it. It won’t work, as it would require a change in exported output that is already in use — so you would need to delete all the stacks that import from it. This is a very common problem, where CloudFormation import/export might be in fact tight coupling.
By using SSM Parameter to pass the SQS name, we can introduce loosely coupled binding. There, only parameter name is immutable, but its value can be changed anytime. Fewer dependencies, more flexibility, and that’s what cloud is all about.
This machine has no brain, use your own.
These CloudFormation limitations are there for a reason, so breaking them is not always the best idea. You should always think if it’s okay to do that so you won’t get your infrastructure in a messy state even when using IaC for everything. Values in Parameter Store can be changed manually and you won’t even notice, so keep your IAM secured and always follow the principle of least privilege. Have your application prepared for that kind of changes, eg. read parameters during container startup and let it fail if something changes.
Parameter Store is so versatile that it makes it to the top of my favourite AWS services. Simple yet powerful and really flexible. It can help you with many different tasks, so don’t bother using it for more than just keeping simple configuration files.
At Nordcloud we are always looking for talented people. If you enjoy reading this post and would like to work with public cloud projects on a daily basis — check out our open positions here.
