How to

Set Up Monitoring for Jina Search Applications

Detect and diagnose problems early and get valuable performance data to optimize and improve your application.

Introduction

Monitoring is a process of observing, tracking, and measuring applications running in production. By monitoring production applications, organizations can identify and resolve issues before they cause customer experience problems or business-critical outages.

In Jina search applications, many different components can be monitored as a Jina Flow exposes several core metrics that allow you to look deeper at what is happening inside it. Metrics allow you to monitor the overall state of your Flow, detect performance bottlenecks, or alert your team when some components of your Flow are down.

To set up monitoring in Jina Seach Applications, we will use the following tech stack:

• Prometheus
• Grafana

What is Prometheus?

Prometheus is an open-source system monitoring and alerting toolkit. Prometheus collects and stores its metrics as time-series data, i.e., metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.

Jina Flows exposes metrics in the Prometheus format. This is a plain text format that both humans and machines can understand. Prometheus is in charge of collecting and storing these metrics and sending them to Grafana for visualization.

What is Grafana?

Grafana is an open-source tool that allows you to query, visualize and understand your metrics from a central dashboard. Prometheus allows external entities like Grafana to access the aggregated metrics via the query language PromQL. It also comes up with the option to create, explore, and share beautiful dashboards with your team so you can have a better overview of your monitoring and alerting setup.

Monitoring Set Up Walkthrough

Now we will walk you through the steps required to set up monitoring for Jina applications using Prometheus and Grafana. The first step in the process is to deploy your Jina Flow on Kubernetes, which will enable the following monitoring features:

• It will allow the Prometheus operator to discover new endpoints and scrape the metrics data automatically.
• It will allow the natural monitoring extension with the rich built-in Kubernetes metrics.

You need to have access to a Kubernetes cluster to follow the rest of this tutorial. If you don’t have one, you can easily spin up a new instance by following the instructions mentioned in the documentation.

The next step is to Deploy Prometheus and Grafana on your k8s cluster, which is as easy as executing the following command:

helm install prometheus prometheus-community/kube-prometheus-stack — set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

Now you can deploy the Flow that we want to monitor on the Kubernetes cluster. Following is a step-by-step guide to do that:

Running this code will create a config folder containing the Kubernetes YAML definition of the Flow. Check out the following documentation for a deep dive into deploying Flows on Kubernetes.

Run the following command in the terminal from your working directory to deploy the Flow that we created above:

kubectl apply -R -f config

In a couple of minutes, you can check the health and status of the pods with the following command:

kubectl get pods

Output of the previous command

Once you verify that the pods are up and running, you can use the following command to spin up the Prometheus instance with the aggregated metrics:

kubectl port-forward svc/prometheus-operated 9090:9090

Output from the previous command

Accessing Grafana Dashboard

To visualize the metrics in real-time, you need to connect your Prometheus instance with the Grafana dashboard. Before moving forward, you have to do a port-forward for querying the gateway. To do that, you can use the following command:

kubectl port-forward svc/gateway 8080:8080

Now, to access the Grafana dashboard, you can run the following command in your terminal and open http://localhost:3000 in your browser.

kubectl port-forward svc/prometheus-grafana 3000:80

Check out the following documentation to get the username and password for accessing the Grafana dashboard.

Once you can access the Grafana homepage, then go to Browse then import and copy and paste the JSON file to bring up the dashboard. But the dashboard will look empty until you query the Flow.

To query a deployed Flow on Kubernetes, you can use Jina Client with the specific port to send and receive the requests. Let’s look at a simple example to query a Flow using Jina client 👉

After running this code, you will be able to view request details updated in real-time on the Grafana dashboard, as you can see in the figure below 👇

Grafana Dashboard!

Congratulations 🎉

You have successfully enabled monitoring for your Jina applications, and it will allow you to monitor all the incoming and outgoing requests in real-time!

Monitoring in action!

To give you a glimpse of the potential capabilities of the monitoring feature, we have created an interactive video walkthrough to enable monitoring for a real-world application. This video will walk you through the monitoring setup for an image search application that uses Totally Looks Like dataset.

Follow along with the interactive video tutorial 👉

📕 For the source code, check out the following Colab Notebook 👇

Jina Monitoring

Learning Resources

• Monitoring Documentation
• Jina’s Learning Bootcamp
• DocArray Documentation

Venture into the exciting world of Neural Search with Jina’s Learning Bootcamp. Get certified and be a part of Jina’s Hall of Fame! 🏆

Stay tuned for more exciting updates on the upcoming products and features from Jina AI! 👋