Python Libraries for DevOps(Day13)

Welcome to Day-13 of your DevOps journey! Continuing our DevOps series, today we will be diving into Python Libraries for DevOps.

In our previous discussion, we covered Intermediate Python Concepts for Automation. If you haven’t read it yet, you can check it out [here].

As a DevOps engineer, efficiency is key, often achieved through leveraging Python libraries that offer pre-built functionality for various tasks. These libraries significantly reduce the amount of code needed to perform essential operations.

Python Virtual Environment
A virtual environment in Python is used to create an isolated environment for your projects, allowing you to manage dependencies and packages separately for each project without conflicts. This ensures that changes in one project’s dependencies do not affect other projects, providing a clean, controlled, and reproducible development setup.

Each Python project may require different versions of Python, different set of libraries, and dependencies. Hence, we use virtual environments to isolate and manage these dependencies separately for each project.

Steps on how to create a virtual environment in Windows 10 using VS Code IDE: First, create a folder in VS Code where all files will reside. Then, name your project and follow these steps in the VS Code terminal to activate it.

Step 1: Install virtualenv if not already installed
First, ensure you have virtual environment installed. You can install
it using pip:

pip install virtualenv

Step 2: Create a Virtual Environment
You can create a virtual environment using the correct Python version reference. Here’s the correct command:

virtualenv -p python3.12 env

Make sure python3.12 is the correct reference to the Python version installed on your system. If it’s not installed, you need to install Python 3.12 first.

Step 3: Activate the Virtual Environment
After creating the virtual environment, you need to activate it. Here’s how you can do it in Windows PowerShell:

.\env\Scripts\Activate

To understand better, heres the link to a python playlist.

Reading JSON and YAML in Python

• As a DevOps Engineer you should be able to parse files, be it txt, json, yaml, etc.
• You should know what all libraries one should use in Python for DevOps.
• Python has numerous libraries like os, sys, json, yaml etc that a DevOps Engineer uses in day to day tasks.

As a DevOps Engineer, being proficient in handling various file formats and configurations is crucial. Here are some Python libraries commonly used in DevOps tasks:

1. os: Used for interacting with the operating system, managing files, directories, to create read and write a file, etc.
2. sys: Provides access to system-specific parameters and functions, useful for command-line arguments and system configuration.
3. json: Essential for parsing and manipulating JSON data, which is commonly used in APIs and configuration files.
4. yaml: Used for parsing YAML files, which are popular for configuration management due to their human-readable format.
5. requests: Ideal for making HTTP requests, interacting with APIs, and retrieving data from web services.
6. docker: Python API for Docker, enabling automation of Docker container management and operations.

These libraries are essential tools for handling various aspects of automation, configuration management, and system administration tasks that are core to the responsibilities of a DevOps Engineer.

Tasks:-

1. Create a Dictionary in Python and write it to a json File.
2. Read a json file services.json kept in this folder and print the service names of every cloud service provider.

output:

aws : ec2
azure : VM
gcp : compute engine

3. Read YAML file using python, file services.yaml and read the contents to convert yaml to json

To run this program properly, we’ll need to add services.json and services.yaml files in the same directory as the parser.py file.

services.json:

{
    "services": {
      "debug": "on",
      "aws": {
        "name": "EC2",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      },
      "azure": {
        "name": "VM",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      },
      "gcp": {
        "name": "Compute Engine",
        "type": "pay per hour",
        "instances": 500,
        "count": 500
      }
    }
  }

services.yaml:

services:
  debug: 'on'
  aws:
    name: EC2
    type: pay per hour
    instances: 500
    count: 500
  azure:
    name: VM
    type: pay per hour
    instances: 500
    count: 500
  gcp:
    name: Compute Engine
    type: pay per hour
    instances: 500
    count: 500

parser.py:

import json
import yaml

json_file = "services.json"
yaml_file = "services.yaml"

with open(json_file, 'r', encoding='utf-8') as f:
    json_data = json.loads(f.read())

print("JSON:\n",json_data)

with open(yaml_file, "r") as stream:
    try:
        yaml_data = yaml.safe_load(stream)
    except yaml.YAMLError as exc:
        print(exc)


print("YAML:\n",yaml_data)

After gathering all the requirements and files, here’s the actual code:

import json
import yaml
import os

# Task 1: Create a Dictionary in Python and write it to a JSON file
data = {
    "services": {
        "debug": "on",
        "aws": {
            "name": "EC2",
            "type": "pay per hour",
            "instances": 500,
            "count": 500
        },
        "azure": {
            "name": "VM",
            "type": "pay per hour",
            "instances": 500,
            "count": 500
        },
        "gcp": {
            "name": "Compute Engine",
            "type": "pay per hour",
            "instances": 500,
            "count": 500
        }
    }
}

with open('output.json', 'w', encoding='utf-8') as json_file:
    json.dump(data, json_file, indent=4)
print("Dictionary has been written to output.json")

# Task 2: Read a JSON file services.json and print the service names of every cloud service provider
json_file = "services.json"
with open(json_file, 'r', encoding='utf-8') as f:
    json_data = json.load(f)

services = json_data["services"]
print(f"aws : {services['aws']['name'].lower()}")
print(f"azure : {services['azure']['name']}")
print(f"gcp : {services['gcp']['name'].lower()}")

# Task 3: Read YAML file services.yaml and convert its contents to JSON
yaml_file = "services.yaml"

# Debug print statement
print(f"Looking for YAML file in: {os.path.abspath(yaml_file)}")

try:
    with open(yaml_file, "r", encoding='utf-8') as stream:
        yaml_data = yaml.safe_load(stream)
        json_data_from_yaml = json.dumps(yaml_data, indent=4)
        print("YAML data converted to JSON:\n", json_data_from_yaml)
except FileNotFoundError:
    print(f"FileNotFoundError: {yaml_file} not found.")
except yaml.YAMLError as exc:
    print(exc)

I hope you learned something new. Thank You for giving it a read.