JSON like templates for automated infrastructure resource provisioning like ARM Templates for Azure or AWS CloudFormation used to be great, but that’s for the past. Both Azure and AWS have introduced higher level SDKs, start using them to save on development and maintenance costs.
Note: This article is focused on Azure but the concept is similar for AWS.
There are various ways that one can prepare a deployment environment in Azure, namely: Azure Resource Manager (ARM) templates, PowerShell, Azure CLI, REST API, Client SDKs like Azure Resource Manager Libraries for .Net or third party tools like Terraform.
In this article I’m comparing ARM templates and Azure Resource Manager Libraries for .Net — which I call it ARM.Net for the sake of brevity .
Before we get to the details of the two technologies, let’s declare the context and the problem that we’re trying to address.
The idea of automating infrastructure preparation is not new, people used to write automation scripts for that reason long before cloud services came to existence. However since the existence of the cloud, the need for the automation is surging. As we are moving away from the bigger monolithic software systems to smaller services, managing the growing number of Servers/VMs/Services manually is getting more and more costly to the point that it becomes impossible.
Infrastructure as Code
Despite the good old automation scripts, Infrastructure as Code (IaC) is a new term which means more than just automation. The word “code” means we need version control, maintainability, reusability, etc. More importantly by IaC we imply that the developers are responsible for building the infrastructure through code and maintain it.
The idea of shifting the deployment responsibility from operations to developers is the reason that a Service Oriented Architecture can finally be implemented through Microservices without pushing too much work to the operations.
As a developer even if you’re not familiar with any of the cloud deployment tools, like Azure ARM template for example, you know how a good code looks like. A good code is readable, reusable, maintainable so is a good infrastructure code.
Here is a sample of an ARM template which creates an Azure Container Registry.
As you can see ARM template is a declarative language in JSON format comparing to the Azure Resource Manager Libraries for .Net (ARM.Net) which isn’t declarative — We’ll see later in this post what benefits being declarative may have.
With a few negligible differences, the two methods are almost the same. While the .Net library is obviously more readable, one can read the JSON version fairly easy too.
Let’s see a more complex example: We need to create an Azure App Service (web app for Linux container) along with a new Service Plan. The following is how your ARM Templates looks like.
First we create the service plan:
Next the App Service:
That’s so many lines of JSON code and almost impossible to read comparing to the 8 lines of Fluent interface of ARM.Net.
ARM.Net is closer to human language and it abstracts away the unnecessary details like docker container settings keys in the Application Settings.
There are other observable benefits too, like you can use it almost with no help from the documentation as it’s pretty much self explanatory. That’s not the case with ARM template at all. ARM template is not just hard to read, it’s also hard to learn. A more complex task like fetching a secret from an Azure Key Vault requires going deep into the documentations and community blogs.
Even if you manage to master the ARM templates, every time you want to create another resource you’ll have to take a peek at the documentation again or copy from one of your previously created ones.
AWS uses the “assembly” and “compiler” terminology to explain the difference between CloudFormation and Cloud Development Kit (CDK) and I think that’s a smart choice of words. Like when back in the day people where actually writing code in assembly language until the compilers were developed.
Today, no one thinks about writing a general application in assembly language for obvious reasons and I think it’s fair to say now that we have Cloud Client SDKs (compilers), let’s stop using the JSON templates (assembly language) unless it’s absolutely needed.
With Client SDKs you can take the SDK and build another object oriented library on top of it specific to your organization needs and save your company so much time and money. Conventions like naming conventions or pricing tiers can be further abstracted. For example a one liner.
CreateWebAppFor(environment: “Production”, name: “MyWebApp”);
- Can take a web application in the current folder,
- build it
- create docker image
- push the docker image to production container registry
- provision the App Service along with a production Service Plan
- setup SSL certificates
- and add prefix/suffix to the name to make sure it’is in line with companies naming convention
*Well maybe I exaggerated a bit but I’m sure you’ll get the point.
ARM template can’t run standalone and it needs PowerShell or Azure CLI for deployment. In general the ARM template is just not capable of doing all that you’ll need for automation. It can’t add a login to an Azure SQL for example or upgrade db schema if needed.
As soon as you start writing a script in PowerShell, you may ask yourself, well why not just doing the whole thing in PS?
PS is very powerful — as the name suggests — which becomes a golden hammer in some cases. I’m not a big fan of writing complex code in PS because it tends to get messy and hard to maintain when the script files grow larger, however I see why one may decide to use it over ARM templates.
Comparing to the ARM template, PS is not as verbose and besides your team doesn’t have to learn ARM templates and you already had to write the PS script to execute the template.
Take it to the team
The hardest part of adapting a new technology is to sell it to your team. With the DevOps methodology it’s even harder. Devs and Ops don’t speak the same language and they have different concerns. I’ll try to answer a few common questions here to help you if you want to take it to your team.
C# is not a declarative language, ARM is
Declarative is when we ask what we want without specifying how. For example in SQL when we say SELECT * FROM MyTable, we don’t care about how the result-set is being populated like if a for-loop logic is there or whether there is an “if branch” somewhere to fetch data from the cache if available — OK sometimes we have to care for the sake of performance, but not in general.
I agree the way we use JSON in ARM templates makes it declarative. Being declarative doesn’t provide any advantages on it’s own. Most of the benefits of a declarative language come from the fact that declarative languages are domain specific and not general purpose, meaning they can effectively do what they meant to do.
C# is an imperative and general purpose language, but it doesn’t mean one can’t create a domain specific library in C# with benefits similar to a declarative language. We use a declarative language for cloud resource provisioning as we don’t want to know how resources are being created in the background. ARM.Net is designed to be domain specific and fluent. it was designed to abstract away the implementation details so we can tell it “what” we want, and not to worry about “the how”.
Now a fluent interface doesn’t make C# declarative but that’s a benefit in my opinion. With a domain specific API in a general purpose programming language, while you get most of the benefits of a declarative language, you still can do everything that a declarative language can’t do for example you don’t need PowerShell to run the C# code, but you do need it for ARM template.
I can export an existing resource group using my azure portal and import it effortlessly
Well you can do that in theory, actually that was my motivation to go with ARM template when a college showed that to me.
Not everything can be exported from an existing resources group and when export issues happen you have to go through the errors one by one and add them back. Even for the fully exportable resources, what you’ll get is a machine generated code which is not considered to be a healthy code because of readability and maintainability issues.
You may be able to get some ideas from an exported ARM template, but you need to rewrite it if you care about code quality. Please don’t fall for another code generator tool from Microsoft.
ARM template takes care of dependencies so I don’t have to bother
Yes it can take care of dependencies but there is no magic and you still need to define the dependencies. In my opinion .Net Libraries has a better syntax for managing the dependencies. It can either accept the dependency as an existing resource or it can create it on the fly like when we created the Web App along with the Service Plan in the above examples.
ARM template can create the resources in parallel
Correct, but no one can stop you from doing the same in C# using parallel programming. Also ARM.Net has an ICreatable interface which you can use to create supported resources in parallel.
ARM templates are idempotent
Being idempotent means you can run the code as many times as needed and only change things that were changed. Simply you “insert if not exists” or “update if exists”. Both ARM template and ARM.Net are idempotent.
Ops can understand ARM templates and can potentially help with the code
I Agree that ops has to understand and give feedback on the infrastructure code. One of the benefits of IaC is that it can be used as a common language and possible as documentation. So it is important that we use a technology that’s understandable by both developers and operations.
Because ARM.Net has a fluent interface, I don’t think it’s any harder to understand than ARM template for someone who’s new to both. For the people who already know ARM template, there may be a bit of learning curve to get familiar with ARM.Net.
Operations may be more comfortable to write templates than programs, but when reading someone else’s code, if the code is not readable, it’s hard to understand. However, when we agreed to move the infrastructure provisioning responsibility from ops to devs, it’s fair to say that we need a language which is easy to develop by developers and easy to read by other stakeholders including operations.
We have tons of ARM templates, we don’t want to throw them away
That’s fair and you shouldn’t. With ARM.Net you can deploy your existing ARM templates using the WithTemplate and WithParameter methods. You’ll use ARM.Net for new resources and whenever you have some time you can refactor the existing ones, just consider it a type of tech dept.
What do Microsoft and Amazon have to say about Client SDKs?
Microsoft stays neutral and recommends to:
“Choose the tools and APIs that work best for you — they have the same capability and provide consistent results.”
“You can think of the CDK as a cloud infrastructure “compiler”. It provides a set of high-level class libraries, called Constructs, that abstract AWS cloud resources and encapsulate AWS best practices. Constructs can be snapped together into object-oriented CDK applications that precisely define your application infrastructure and take care of all the complex boilerplate logic. When you run your CDK application, it is compiled into a CloudFormation Template, the “assembly language” for AWS cloud infrastructure.”
We are living in a period of time that the technology is improving rapidly. What used to be a good solution a few years ago may not be the best one today. The competition between tech companies is so fierce that small improvements over the software development process can make a big difference just as a tenth of second can make a big difference in a race. Adapting Client SDKs over JSON templates is one those small changes that can potentially have a big impact on you organization’s costs and time to market.