Apply Graph Theory to Oracle Cloud Infrastructure with Python + NetworkX + Cytoscape

This article is a follow-up to the previous article that outlined the ability to use Oracle Cloud Infrastructure Search + Asyncio Python + Oracle Data Visualization to Accelerate tenancy visibility

Accelerate Tenancy Visibility with Oracle Cloud Infrastructure Search + Asyncio + Parallel -Python…

As Cloud Engineers we are on the lookout to understand everything about the tenancy we administer in great detail. This article is about how to leverage NetworkX + Cytoscape + Python to graph OCI Infrastructure and visualize infrastructure and it’s interdependencies.

For the Impatient:

vamsiramakrishnan/oci-search

Sample Graph of a single Compartment

The Compartment is the umbrella under which all resources exist
Vnics are attached to instances and belong to a subnet
Subnets are a part of VCN
Security Lists are attached to Subnets 
Security Lists are also part of VCNs
Route Tables while belonging toa VCN are attached to a Subnet

and

The Big Picture

Here is a snapshot of how Graphing the Entire tenancy information looks like.

Image of Tenancy Graph

Tenancy Statistics:

Number of Cloud Users : 80+ 
Number of Cloud Compartments : 100 + Compartments
Number of Regions Subscribed : 3 Regions
us-phoenix-1
eu-frankfurt-1
ap-mumbai-1
Number Resources : 1126
Number of Attributes per Resource: 20-25Some of the Services Used : 
- OCI Compute 
- OCI Block Storage
- OCI Virtual Cloud Networks
- OCI Subnets
- OCI Route Tables
- OCI Security Lists
- OCI Local Peering Gateways
- OCI Load Balancers
- OCI Object Storage
- OCI Notifications Service
- OCI Database Cloud Service
- OCI Streaming 
- OCI Oracle Kubernetes Engine
- OCI Functions 
- OCI API Gateway 
- OCI Analytics
- OCI Integration 
- OCI Content and Experience 
- OCI Oracle Digital Assistant

When loaded into NetworkX as a graph led us to have

Name: Tenancy-Graph
Type: DiGraph
Number of nodes: 1494
Number of edges: 2650
Average in degree:   1.7738
Average out degree:   1.7738
Network density: 0.001188052051922806

Motivation

Cloud Infrastructure is a natural Directed Multi-Graph

1. Every Resource is a Node
2. How Resources are related would be an Edge
3. Standalone Properties of Resources could be Node Attributes
4. Attributes defining the relationship would be Edge Attributes

Multiple Relationships
A Resource could be related to another resource in multiple-ways
Directed
Cloud Infrastructure relationships are Directed, heirarchies of order exist.

While Cloud Infrastructure is a natural Graph, the access patterns make resource characteristics feel this way

1. Every Resource is a JSON Document Identified by a unique OCID
2. The Search APIs serve as an *OCID index 
3. JSON Documents have several resource attributes , which contain OCIDs of other resources which symbolify links to other resources

*OCID — Oracle Cloud Identifier

Unique Resource ID for a given tenancy 

The Process

1) Use Search to fetch Resource OCIDs
2) Use OCIDs to Fetch Detailed Info
3) Populate tenancy Dictionary
4) Populate Relationships
5) Write GraphML File
6) Open it in Cytoscape

Sample GraphML File

The Code

Jupyter Notebook Gist

Toolchain

Jupyterlab

Project Jupyter

Next Generation Notebook tool to execute Python Code with Terminal , Markdown, and significant serverside capabilities

NetworkX

Overview of NetworkX - NetworkX 2.4 documentation

Python Library to create Graphs from JSON 
Easy to use Familiar Python Dictionary Syntax

Cytoscape

Cytoscape

One among the several Graph Visualization Tools like 
Gephi | D3 | Dato |