How to use Loki from Grafana Cloud with Openshift Network Observability
In this blog post, you will see how to use Loki product from Grafana Cloud to setup quickly OpenShift Network Observability operator to play with this great operator.
With the Openshift version 4.12 Red Hat introduced the new release of Openshift Network Observability operator to observe the network traffic for OpenShift Container Platform clusters. The Network Observability uses the eBPF technology to create network flows.
NetObserv
The Network Observability operator called internally "NetObserv" offers the visualization of the intra-cluster traffic. Our great development Team created a eBPF agent to hook into the network events so it can capture and export data coming in and out of the interfaces of the kernel level. eBPF provides a powerful and performant agent and it's agnostic on a particular CNI (Container Network Interface).
In the high level architecture diagram of Network Observability coming from this great source [1] we can see the role of Loki. Loki is a prerequisite for using the Network Observability Operator.
I didn't want to install Loki on my cluster as shown in this great blog post from Tommer Amber, but I prefered to consume directly the Loki instance offered by my Grafana Cloud Free plan.
As a summary of our goals, we want:
- To install Network Observability operator
- To create a Grafana Cloud account.
- To create a FlowCollector using Loki instance URL from Grafana Cloud.
Part 1 : Install Network Observability operator
From your Openshift Web UI, install the Network observabilityoperator
Part 2: Create Grafana Cloud account and using Loki API key
From Grafana.com, create a free account (no credit card required) if you don't have it. Once done click on My account (on the top right of the menu of grafana.com).
Select Loki part and click on the blue button Send Logs, a page will appear with all informations needed to integrate Loki with our Network Observability operator.
The format of the Loki URL needed for Network Observability is something like :
https://<user>:<Your Grafana.com API Key>@<URL>
example : https://341370:<Your Grafana.com API Key>@logs-prod-013.grafana.net
where 341370 is the User, logs-prod-013.grafana.net is your loki instance on Grafana Cloud.
To generate your API Key, the Grafana interface propose a Generate now link. Capture all this values, you will need them for the last part.
Part 3: Create a FlowCollector using Loki from Grafana Cloud
From your Openshift Web UI, navigate to: [NAMESPACE==netobserv] Administrator view -> Operators -> Installed Operators -> NetObserv Operator -> Create FlowCollector -> Go to loki section -> replace url value with url created in Part 2.
example : https://341370:eyJrIjoiN2Y2Y2UzZjg1ZmRlN2M4ODdkMmQ3YWFkMGYwYjQ2ZDJhZDg0ZTQ0NCIsIm4iOiJvY3A0IG5ldG9ic2VydiIsImlkIjo3NTc4YTL@logs-prod-013.grafana.net/
Additional resources :
- Network Insights in a Distributed Environment
- HowTo install Loki [S3 persistent storage] & NetObserv Operator on OpenShift 4 (4.10–4.12)
- A guide to Providing Insight with Network Observability.
- Check Out the new Network Observability Support in Openshift 4.12
- How to Display Your Kubernetes/OpenShift Network Flows