Exploring the LinkerD Dashboard[via Ingress] with Essential Commands

gaurav agnihotri
7 min readMay 23, 2023

--

LinkerD DashBoard Look

How to access the Dashboard —

We will discuss Two methods for accessing the Dashboard.”

Via Command-

Run the below command to navigate to LinkerD Dashboard

# linkerd viz dashboard

Linkerd dashboard available at:
http://localhost:50750
Grafana dashboard available at:
http://localhost:50750/grafana
Opening Linkerd dashboard in the default browser

The above command redirects to your default browser and shows you the LinkerD dashboard like below.

Via Ingress —

Instead of accessing the linkerd viz dashboard every time you want to check the status, you have the option to expose the dashboard through an ingress.

Use Internal Ingress (means Where Ingress controller is using an internal Load balancer)

apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: web-ingress-auth
namespace: linkerd-viz
data:
auth: <username-password>
---
# apiVersion: networking.k8s.io/v1beta1 # for k8s < v1.19
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-ingress
namespace: linkerd-viz
annotations:
nginx.ingress.kubernetes.io/upstream-vhost: $service_name.$namespace.svc.cluster.local:8084
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Origin "";
proxy_hide_header l5d-remote-ip;
proxy_hide_header l5d-server-id;
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: web-ingress-auth
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
spec:
ingressClassName: nginx <<<<<<<< Internal Ingress/Internal Loadbalancer
rules:
- host: dashboard.gaurav.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 8084

DashBoard Tour —

When you first enter the dashboard, you will start out in the top-level namespaces view.

The namespaces page already provides a comprehensive set of information.

At the namespace level, you can obtain an overview and all the required data.

For more detailed insight, navigate to the specific namespace associated with your application.

In our case, emojivoto [NameSpace].

Now Go a little deeper into one of the deployments and see the stats.

Let’s Choose “emoji” deployment-

We get the stats using two ways, one via browser and one via command line-

Via browser —

Namespace>emojivoto>deployment/emoji, and See the Stats like

SR(Success Rate), RPS(Request per second), P99(Latancy)

Via command Line —

linkerd viz stat deploy -n emojivoto

Let’s break down the example and explain it in simpler terms:

We have four different components [deployments] or services in the Emojivote setup:

  • emoji
  • vote-bot
  • voting
  • web

Here’s what each column means:

NAME       MESHED   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99   TCP_CONN
web 1/1 92.65% 2.3rps 2ms 3ms 3ms 3

- NAME: The name of the component or service.
- MESHED: It indicates that all containers of the component are part of the service mesh (`1/1` means all containers are meshed).
- SUCCESS: The percentage of successful requests or operations for that component.
- RPS: The average number of requests per second (rps) handled by the component.
- LATENCY_P50: The median or middle value of latency experienced by the requests (in this case, it’s 1 millisecond).
- LATENCY_P95: The maximum latency experienced by 95% of the requests (in this case, it’s 3 milliseconds).
- LATENCY_P99: The maximum latency experienced by 99% of the requests (in this case, it’s 3 milliseconds).
- TCP_CONN :The number of TCP connections made by the component.

Now, let’s take the `web` component as an example to understand these metrics. We see that it is meshed (`1/1`), indicating that all its containers are part of the service mesh. It has a success rate of 92.65%, which means 92.65% of its requests or operations were successful.

The `web` component handles an average of 2.3 requests per second (rps). The latency percentiles tell us about the response time. The `LATENCY_P50` is 2 milliseconds, which means half of the requests handled by the `web` component have a latency lower than 2 milliseconds.

The `LATENCY_P95` and `LATENCY_P99` are both 3 milliseconds. It means that 95% and 99% of the requests, respectively, had a latency lower than 3 milliseconds. In other words, only a small percentage of requests experienced higher latency.

Lastly, we see that the `web` component has 3 TCP connections established.

By monitoring these metrics, we can get insights into the performance of each component in our system, such as success rate, average requests per second, and latency distribution. This information helps us identify any performance issues, bottlenecks, or anomalies and take appropriate actions to optimize the system’s performance.

Essential Commands —

Linkerd viz edges -

linkerd viz edges deploy -n emojivoto

In Linkerd’s visualization (viz) component, “edges” represent the connections or communication pathways between different services within a service mesh.

By observing the edges in Linkerd viz, you can gain insights into the interactions and dependencies between services within your service mesh.

The output of the linkerd viz edges command provides information about the connections or edges between different services within the service mesh. Let's break down the output:

SRC (Source): This column indicates the source service or component initiating the communication.

DST (Destination): This column indicates the destination service or component receiving the communication.

SRC_NS (Source Namespace): It represents the namespace of the source service.

DST_NS (Destination Namespace): It represents the namespace of the destination service.

SECURED: The checkmark (√) in this column indicates that the communication between the source and destination services is secured.

Linkerd viz ‘tap’ -

In Linkerd, the linkerd viz tap command is used to capture and observe live traffic flowing through the service mesh. It allows you to tap into specific requests and responses between services.

  • By using the linkerd viz tap command, you can:
  1. Debug issues: You can tap into specific requests to diagnose and troubleshoot issues in real time. By inspecting the traffic.
  2. Monitor traffic: It enables you to observe and analyze the traffic flowing between services. You can capture and analyze metrics related to latency, success rates, error rates, and other relevant data.

Namespace > emojivoto > deployment/emoji

LinkerD tap

After Clicking on “tap” -

“Tap” in deep

Tap- Tapping is similar to recording a phone call, and in Linkerd, it involves capturing the traffic flow and activities.

You can view tap data in two ways: through the dashboard, as explained earlier, or by using the command-line interface.

If you prefer, you can copy the command from the dashboard and paste it into your terminal.” [Check the above screenshot]

linkerd viz tap deployment/web --namespace emojivoto --to deployment/emoji --to-namespace emojivoto --path /emojivoto.v1.EmojiService/ListAll
linkerd viz tap deployment/web --namespace emojivoto --to deployment/emoji --to-namespace emojivoto --path /emojivoto.v1.EmojiService/ListAll

req id=107:0 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :method=POST :authority=emoji-svc.emojivoto:8080 :path=/emojivoto.v1.EmojiService/ListAll
rsp id=107:0 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :status=200 latency=1862µs
end id=107:0 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true grpc-status=OK duration=34µs response-length=2140B
req id=107:1 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :method=POST :authority=emoji-svc.emojivoto:8080 :path=/emojivoto.v1.EmojiService/ListAll
rsp id=107:1 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :status=200 latency=1885µs
end id=107:1 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true grpc-status=OK duration=44µs response-length=2140B
req id=107:2 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :method=POST :authority=emoji-svc.emojivoto:8080 :path=/emojivoto.v1.EmojiService/ListAll
rsp id=107:2 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :status=200 latency=1889µs
end id=107:2 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true grpc-status=OK duration=41µs response-length=2140B
req id=107:3 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :method=POST :authority=emoji-svc.emojivoto:8080 :path=/emojivoto.v1.EmojiService/ListAll
rsp id=107:3 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :status=200 latency=1790µs
end id=107:3 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true grpc-status=OK duration=114µs response-length=2140B
req id=107:4 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :method=POST :authority=emoji-svc.emojivoto:8080 :path=/emojivoto.v1.EmojiService/ListAll
rsp id=107:4 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :status=200 latency=1874µs
end id=107:4 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true grpc-status=OK duration=117µs response-length=2140B
req id=107:5 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :method=POST :authority=emoji-svc.emojivoto:8080 :path=/emojivoto.v1.EmojiService/ListAll
rsp id=107:5 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true :status=200 latency=1813µs
end id=107:5 proxy=out src=x.x.x.x:53250 dst=y.y.y.y:8080 tls=true grpc-status=OK duration=34µs response-length=2140B

Error Fix —

if you find any permission error on the above command, Fix it by using the below command-

>> Not recommended for production, only do this for testing/development.

kubectl create clusterrolebinding \
$(whoami)-cluster-admin \
--clusterrole=cluster-admin \
--user=$(gcloud config get-value account)

More details related to permission you will get here —

Linkerd Tap Permission —

Linkerd viz ‘top’ -

linkerd top command is used to monitor and display real-time metrics and statistics for the services running within the service mesh. It provides a quick overview of resource utilization and the performance of services.

You can view tap data in two ways: through the dashboard and via the command line.

linkerd viz top deployment/emoji --namespace emojivoto
LinkerD TOP Dashboard

Via Command —

LinkerD top Command line

I hope this post is informative and useful for you :)

If you enjoy the blog, please give me a Clap : ) and Follow me for more such content.

Crafting these articles demands countless hours of ideation, research, and writing. This year has seen me invest over 500 hours into this craft alone. If my work has brought you joy, would you kindly consider supporting me with a coffee? Your gesture would mean the world to me. If not, thank you dearly for your readership. ❤️

Buy-me-a-coffee ❤️

--

--