Hashicorp has developed a new tool for customizing the way we build images. Previously we would be using various 3rd party tools with no coherency and type to build machine images like in iso format to keep a healthy backup of machine state.
So how can we do that better?
Packer is one of the tools used in today's cloud industry to bundle and package machine images or what AWS Folks call as AMI — Amazon Machine Image, Google Compute Guys — gccompute image or Azure VM Images. Today we will learn about how to make AMIs and on the next two tutorial, I will give you guys a walkthrough for the Google Cloud and Azure usage of Packer.
So what is AMI and what the heck is Packer?
Packer is an open source tool for creating identical machine images for multiple platforms from a single source configuration
Packer is lightweight, runs on every major operating system, and is highly performant. It can create machine images for multiple platforms in parallel. Packer does not replace configuration management like Chef or Puppet. In fact, when building images, Packer is able to use tools like Chef or Puppet to install software onto the image.
A machine image is a single static unit that contains a pre-configured operating system and installed software which is used to quickly create new running machines. Machine image formats change for each platform. Some examples include AMIs for EC2, VMDK/VMX files for VMware, OVF exports for VirtualBox, etc.
Suppose you have a system that is running Ubuntu 18.04 and you have done some installing of packages into it with correct version and stuff. You have edited a few configurations on nginx/Apache and made the OS as per your software needs. Now when a DevOps guy actually deploys it on a web instance suppose EC2, EBS on AWS the environment is no more the same. Packages are different in the system, versions are clashing. The setup configs are different, web servers are failing.
How can you correct it?
You will say, I will SSH/RDP into the system configure it and then it will work. Yes, it will, but what if your network traffic increases and you need to use load balancers and your instances get created on their own and you don't have time to manually configure each?
Why Use Packer?
Pre-baked machine images have a lot of advantages, but most have been unable to benefit from them because images have been too tedious to create and manage. There were either no existing tools to automate the creation of machine images or they had too high of a learning curve. The result is that, prior to Packer, creating machine images threatened the agility of operations teams, and therefore aren’t used, despite the massive benefits. Packer brings pre-baked images into the modern age, unlocking untapped potential and opening new opportunities.
So how and when to use Packer?
Once you have made the necessary changes on your system note it down. Write up a packer template/blueprint about the changes or make your DevOps guy write it for you. Once you are done this template will create an exact replica of machine image as your local system where you did the development. Gone are the days when you could say it worked on my machine and Idk why it's not working on Production.
Advantages of Using Packer
- Super fast infrastructure deployment: Packer images allow you to launch completely provisioned and configured machines in seconds, rather than several minutes or hours.
- Multi-provider portability: Because Packer creates identical images for multiple platforms, you can run production in AWS, GCP, Azure, staging/QA in a private cloud like OpenStack, and development in desktop virtualization solutions such as VMware or VirtualBox. Each environment is running an identical machine image, giving ultimate portability.
- Improved stability: Packer installs and configures all the software for a machine at the time the image is built. If there are bugs in these scripts, they’ll be caught early, rather than several minutes after a machine is launched.
- Greater testability: After a machine image is built, that machine image can be quickly launched and smoke tested to verify that things appear to be working. If they are, you can be confident that any other machines launched from that image will function properly.
What are you waiting for? Let’s get started!
The best way to get started is to install Packer and run scripts manually on your local machine to create an AMI.
Follow the steps below in order to build the AMI starting from a vanilla Ubuntu 18.04 image.
Step 1: Prep your machine
- Have your security credentials handy to authenticate to your AWS Account. [AWS Credentials documentation.]
- Install the version of Packer based on the OS of the machine from which you plan to build the image. [Packer docs]
- Open this link(https://www.packer.io/downloads.html)
- If you are in windows download the file and extract the packer.exe and save it in a folder named Packer in your C: Drive (C:\Packer\packer.exe)
- Add it to your PATH Variable by going to your settings from Control Panel. Click on new in the Environment Variables and add the Absolute Path to the file there.
- If you are on UNIX/Linux based system extract and save as in the above step and write the following in your Terminal to save it in the path
export PATH= PATH="$PATH:/your/path/packer"
Step 2: Packer template creation
- Packer uses a JSON template that contains build instructions. The basic construct is one or more builders and multiple steps of provisioners and post-processors. You can read more about templates in the Packer docs and create your own template.
- If you do not have your own Packer scripts, please follow the below gists to get yours.
- base_ami.json is the packer template :
- base_script.sh is the shell script that will install packages inside the machine
- We are using a file-based provisioner that copies the file into the build machine and executes it. We also have a manifest post-processor that enables us to output the AMI ID that was just built.
- In base_ami.json, replace the variable wildcards (security group, subnet id, instance information etc.) with your desired values
- You can validate your template by using packer validate <filename>(ttps://www.packer.io/docs/commands/validate.html)
Step 3: Build AMI
If you have used a vars file like here, use the following command
packer build -debug -var-file=vars.json base_ami.json
or else you can type these
packer build -debug base_ami.json
- Verify on AWS if the AMI was built.
What’s the -debug flag?
You must be thinking whats the -debug flag that we just added to the command there. The debug flag disables parallelization and enables debug mode. The Debug mode flags the builders that they should output debugging information. The exact behavior of debug mode is left to the builder of which you are generating the image(EBS/VB etc). In general, builders usually will stop between each step, waiting for keyboard input before continuing. This will allow the user to inspect the state and so on.
So that’s the basic idea on how to get started with Packer and its Image making power. Check out more at (https://www.packer.io) and stay tuned for next stories on various provisioners and deploying methods of Packer on GCP, Azure.
On a serious end note, DevOps is an immerging field with new technologies coming in day and night and you need to keep up with the pace as a Kubernetes engineer who got paid 20LPA won't be paid tomorrow after AWS EKS is full-fledged.