Using Libcloud to list virtual machines across Azure and AWS

Libcloud is a Python library for interacting with multiple cloud providers using a unified API. Version 1.0.0 was released yesterday, so I thought I would give it a try and list virtual machines (VMs) from Azure and Amazon Web Services (AWS). It was easy and quick. However, I did end up taking quite a few notes, especially while setting things up. So thought of sharing those in this article.

Output

The program lists three virtual machines (demo1, demo2 from AWS and shonsoft from Azure) with their state (running or stopped).

Listing virtual machines across Azure and AWS using Libcloud

Code

Essentially, there are three steps :

  1. Using the get_driver() method to obtain a reference to the cloud provider driver
  2. Instantiating the driver with the credentials to access the cloud
  3. Using the list_nodes() method to list the VMs

The five variables (marked bold) will change based on your specific setup.

# Program to use Libcloud to list VMs from Azure and AWS
# Shon Shah — Jun 23, 2016
# AWS related variables
AWS_ACCESS_KEY_ID = ‘AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY = ‘wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
AWS_REGION = ‘us-west-2
# Azure related variables
AZURE_SUBSCRIPTION_ID = ‘xxxxxxxx–xxxx-xxxx-xxxx-xxxxxxxxxxxx
AZURE_MANAGEMENT_CERT_PATH = ‘C:/Demo/azure_cert.pem’
AZURE_CLOUD_SERVICE_NAME = ‘MyCloudService
# Import
from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver
# List AWS virtual machines
cls = get_driver(Provider.EC2)
driver = cls(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, region=AWS_REGION)
nodes = driver.list_nodes()
print (‘Listing AWS virtual machines…’)
for node in nodes:
print (node.name, node.state)

# List Azure virtual machines
cls = get_driver(Provider.AZURE)
driver = cls(subscription_id=AZURE_SUBSCRIPTION_ID, key_file=AZURE_MANAGEMENT_CERT_PATH)
nodes = driver.list_nodes(AZURE_CLOUD_SERVICE_NAME)
print (‘Listing Azure (classic) virtual machines…’)
for node in nodes:
print (node.name, node.state)

Setup

As I mentioned, setting things up took a little more time and effort because of the hunting and troubleshooting involved. So, I will be a bit more elaborate in describing these steps. I am assuming that C:\Demo is the working directory and you have copied the above code as demo.py there.

1. Azure

Ensure that you have an Azure account with few VMs already created. Please be sure to deploy these VMs using the classic deployment model. I could not find Libcloud support for VMs deployed using Azure Resource Manager (ARM) deployment model.

Create a management certificate that the code can use as a credential to access your Azure account. You can use OpenSSL for this as described below.

Download and install OpenSSL for Windows.

Create a self-signed certificate by running the following two commands in the bin directory of the OpenSSL install location. For me, it was C:\Program Files (x86)\GnuWin32\bin.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout azure_cert.pem -out azure_cert.pem -config “C:\Program Files (x86)\GnuWin32\share\openssl.cnf”
openssl x509 -inform pem -in azure_cert.pem -outform der -out azure_cert.cer

Please be sure to specify the -config parameter. Otherwise, you will get the following error: Unable to load config info from /usr/local/ssl/openssl.cnf.

Upload the certificate (.cer file) to Azure using the Azure classic portal.

Move the private key (.pem file) to C:\Demo.

2. AWS

Ensure that you have an AWS account with few VMs already created.

Get access key ID and secret access key that the code can use as a credential to access your AWS account.

Libcloud also supports using temporary security credentials which provides better security due to the auto-expiring nature of credentials. I did not use those to keep things simple.

3. Libcloud

Ensure that you have downloaded and installed Python 3.5.1. Please be sure to check the Add Python 3.5 to PATH checkbox when installing Python. Other versions of Python supported by Libcloud should also work, but I used 3.5.1

Install Libcloud 1.0.0 using the following command:

pip install apache-libcloud==1.0.0

When establishing a secure connection to a cloud provider endpoint, Libcloud verifies server SSL certificate. For this, it needs to know the location of Certificate Authority (CA) cert bundle. Download the CA cert bundle and copy it to C:\Demo. Then set the SSL_CERT_FILE variable as follows:

SET SSL_CERT_FILE=C:\Demo\ca-bundle.crt

If no valid CA certificates are found, you will get the following error: No CA Certificates were found in CA_CERTS_PATH.

Summary

Libcloud provides a unified Python API to interact with multiple cloud providers. It was easy to list VMs from Azure as well as AWS with few lines of Python code and couple hours of focused effort.

Hope this helps.

— Shon

Like what you read? Give Shon Shah a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.