Creating VM Images with Packer in Microsoft Azure

If you’ve ever found yourself in the situation of needing to create and maintain custom machine images, you probably already know there are a number of different ways to go about it. Furthermore, once created, these images can be deployed in containers and customized, which can be unwieldy work.

In this post, I’ll dive into some very specific reasons for why I like to use Packer for this task, and we’ll also take a look at automating the creation of machine images with BitBucket Pipeline

Packer is an open source tool by HashiCorp for creating machine images, allowing you to ensure all machine images meet a baseline standard as well as speeding up provisioning time for application instances.

Why Use Packer?

• Packer makes it quick and easy to automate the creation and customization of machine images.
• Packer installs and configures software at build time. It checks for bugs in installation scripts at the time the image is built which reduces provisioning time of virtual machines, improves stability and testability.
• Allows us to use a single configuration file to create custom images for multiple platforms.
• Embraces modern configuration management.
• Encourages use of automated scripts to install and configure the software within your Packer-made images.

Installing Packer

Packer is contained in a single binary packer. To create an image with Packer, download and install Packer in one of the following ways:

• Download Packer binary for macOs, Linux, or Windows
• Install using Homebrew by executing brew install packer
• Install using apt-get by executing apt-get install packer

Packer has the following options:

Things to Know

Template: Templates are the configuration files for Packer Images written in JSON format. Thepacker build command runs the builds defined in the template, creating the custom images.

Builders: builders is an array of objects that Packer uses to generate machine images. Builders create temporary Azure resources as Packer builds the source VM based on the template. Learn more about Builders from the Packer documentation, here.

Provisioners: Provisioners can be used to pre-install and configure software within the running VM prior to turning it into a machine image. There can be multiple provisioners in a Packer template.

For example, in the below code snippet there are three types of provisioners: shell, file, and powershell.

Packer has many provisioners. Each provisioner has it’s own configuration options. Check out this Packer Documentation section to learn more about each provisioner and its use case.

Communicator: Communicators allow packer template to communicate with the remote host to build images. Consider them as the “transport” layer for Packer builders to execute scripts, upload files etc. Packer currently support three communicators :

• None
• SSH
• WinRM

Configuration options for each communicator can be found here.

Variables: Variables block contains any user provided variables that packer uses to create image. You can parameterize the packer template by configuring variables from the command line, storing them in a separate JSON file, or environment variables.

Packer gives some guidance how configuration templates work. For example, calling these three variables in builder sections as:

{
"client_id": "{{user `client_id_app1`}}",
"client_secret": "{{user `client_secret_app1`}}",
"tenant_id": "{{user `tenant_id_app1`}}",
}

Parameterizing variables can be useful in these ways:

• A shortcut to values you use multiple times
• To make the template portable
• Default value with an empty string can be overridden at build time
• Allows keeping secret tokens and environment-specific values out of templates

Read more on Packer variables here.

Running Packer

Packer builds the images using the build sub-command followed by a JSON template.

packer build template.json

To validate the template syntax is correct before building the image, run the validate sub-command.

packer validate template.json

Setting Variables:

1. Using Command Line: Set a variable value from the command line by using the -var flag. You can specify the -var flag as many times as required but if you define the same variable more than once, the last value of variable will stand.

packer build \
 -var 'subscription_id=YOUR_AZURE_SUBSCRIPTION_ID' \
 -var 'tenant_id=YOUR_TENANT_ID' \ 
 -var 'client_id=YOUR_CLIENT_ID' \
 -var 'client_secret=YOUR_CLIENT_SECRET' \
 template.json

2. Using a File: You can also set variable values using a JSON file. For example, create a file named variables.json.

Specify variables.json file using -var-file flag such as:

packer build \
 -var-file=variables.json \
 template.json

You can also combine -var and var-file flag to specify values from both the command line and a file.

3. Using Environment Variables: The env function is used for environment variables to set default values of user variables. For example, to set a default value of subscription_id from environment variableMY_AZURE_SUBSCRIPTION_ID:

{
  "variables": {
    "subscription_id": "{{env `MY_AZURE_SUBSCRIPTION_ID`}}",
  }
}

Automation using Bitbucket Pipeline

Wiring your Packer configuration into a CI/CD Pipeline can take your maintenance of machine images to another level, through automation.

Bitbucket Pipelines is one such CI/CD solution, built right into Bitbucket, that requires minimal management effort. It lets your team easily build, test and deploy from your Bitbucket repository. There is no need to setup a CI server or to synchronize repositories in Bitbucket pipeline.

All you need to do is enable the pipeline and commit the pipeline definition to your repository.

First, let’s enable pipelines:

Turn on “Enable pipelines” in Pipeline Settings

Next, configure bitbucket-pipelines.yml and push it into the root of your repository. You can use your own docker image with pre-installed Azure CLI and Packer. For example, a bitbucket-pipelines.yml file to run packer build may look like:

To run the custom job:

1. Go to the Branches view in Bitbucket.
2. Click on the Actions menu for the branch you want to run a pipeline for, then click Run pipeline for a branch:

3. Choose a pipeline, then click Run:

Bitbucket Pipeline can be used to generate new machine images for multiple platforms on every change to your master repository. You can also manually trigger a “custom job”. Workflow of creating a custom image using packer and Bitbucket pipeline will look like:

Configuring Azure for Packer

Now that we’ve got the build automated, let’s deploy our images to an Azure Resource Group.

Packer automatically creates the temporary Azure resources required to build the source VM. But you must define a resource group to capture that source VM. The output (artifacts) from the Packer build process are stored in this resource group.

1. Create Azure Resource Group: Microsoft Azure allows you to create and manage your Azure resources using Azure PowerShell, Azure CLI, Azure Portal and Terraform. Based on your organization’s need you can decide how you want to allocate resources to resource groups. For example, creating a resource group using Azure CLI will look like:

az group create -n myResourceGroup -l eastus

2. Create Azure credentials: Packer authenticates resource groups using a service principal. To create a service principal with az ad sp create-for-rbac and output the credential required for packer build:

az ad sp create-for-rbac --query "{ client_id: appId, client_secret: password, tenant_id: tenant }"

★ See the following Microsoft Doc on using Azure Portal to create an Azure Active Directory application and service principal:

Create identity for Azure app in portal

Packer Workflow in Azure

Builders and provisioners defined in the packer template, carry out the actual build process. Packer has azure-arm builders for Azure which allow you to define Azure resources, such as “managed_image_resource_group_name”, and service principal credentials, created in the preceding step.

Summary

Overall, I think that Packer is a pretty amazing tool that really takes the hassle out of creating identical machine images and provisioning responsibility away.

In this post, we looked at the Packer template and provisioners in order to install software onto the machines prior to turning them into images, configuring Bitbucket pipeline, and steps to configure Packer in Azure.

You can use Packer-made images alongside existing deployment workflows to deploy apps to your VM. As a next step, consider using Terraform, PowerShell, Azure CLI or Azure Portal as tools for provisioning new infrastructure with images generated by Packer.