Neo4J is an ACID compliant graph database that leverages index-free adjacency to perform fast running queries. If you use Kubernetes for orchestrating your cluster and managing your deployments, it can prove helpful to visualize it as a graph.
The Kubernetes Dashboard is the most commonly used tool find out details about the cluster, like services, pods, deployments, resource usage etc. Modeling the cluster definition as a graph can help in visualizing the cluster in a better way. Additionally, it can be used to answer questions such as:
> How many pods does a deployment have, and are all of them “ready”?What are the pods running on a given node (i.e machine)?
> What are the pods that got restarted in the last X hours?
> If all instances of a deployment crash or becomes unavailable for some reason, what other deployments could be facing a loss of service (due to dependancies).
So, how to actually do this?
Fortunately, Kubernetes provides an extensive RESTful interface to extract most of the information we need in JSON format. We can then use a library provided by Neo4J called APOC that allows for easily uploading data directly into a running instance.
The easiest way to access the kubernetes API is by running a proxy using the kubectl tool. You can read more about it here. If your kubeconfig is setup correctly, running the proxy makes the data available via localhost:8080 without having to supply a bearer token.
#Create the POD nodes
CALL apoc.load.json("http://localhost:8080/apis/apps/v1beta1/namespaces/my-namespace/pods", "items", {}) YIELD value as v
WITH v.metadata.labels.app as appName1, v.metadata.labels.name as appName2,v
WITH CASE WHEN appName1 is null then appName2 else appName1 END as appName,v
WITH appName,v.metadata.name as pod_id, v.status.phase as status, v.status.startTime as startedAt, v.spec.nodeName as nodeName
MERGE(p:Pod{id: pod_id}) ON CREATE set p.appName=appName , p.status = status, p.startedAt = startedAt, p.nodeName=nodeName
#Create 'Deployment' nodes
CALL apoc.load.json("http://localhost:8080/apis/apps/v1beta1/namespaces/my-namespace/deployments", "items", {}) YIELD value as v
WITH split(v.metadata.name,"-")[0] as deployment_id, v.spec.replicas as replicas
MERGE(d:Deployment{id: deployment_id}) ON CREATE set d.replicas=replicas
#Create the relationships.
MATCH (p:Pod) where p.appName is not null
MATCH (d:Deployment) WHERE toLower(d.id)=toLower(p.appName)
MERGE (d)-[r:HAS_POD]->(p)#Establish dependancies between deployments:
MATCH(d1:Deployment{id:'depl1'}),(d2:Deployment{id: 'depl2'})
CREATE (d1)-[:DEPENDS_ON]->(d2)
Now Once we have the data, we can start exploring it in the Neo4J browser.
Exploring Data
Find any crashed pods (Status not ‘Running’):
MATCH (n:Deployment)-[]→(p:Pod)
WHERE p.status <> "Running"
RETURN p,n LIMIT 25How many Pods started up on 9/28
MATCH (p:Pod) where p.startedAt CONTAINS "2018-09-28" RETURN pHow many Pods started after 9/28
MATCH (p:Pod) where p.startedAt > "2018-09-28" RETURN count(p)As a next step this information could be used to orchestrate a recycle of running pods . Using the same RESTful API, scaling of our deployments can be achieved as well. Also, by using one of the various visualization tools such as d3.js, cytoscape.js etc. we can build our own visual dashboard and be in control of our cluster.
