Distributed load testing with Kubernetes and Predator

In this tutorial, we are going to use an open-source performance platform designed for APIs crafted at ZooZ (a PayU company) called ‘Predator’.
Predator is a distributed performance testing platform that uses Kubernetes abilities to schedule a distributed load on your APIs which is aggregated to one report in real-time.

Deployment

helm repo add zooz https://zooz.github.io/helm/helm install medium-tutorial zooz/predator --set persistence.enabled=true

If everything went well you should get the following output:

1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods -l "app=medium-tutorial-predator,release=medium-tutorial" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:8080 to use your application"kubectl port-forward $POD_NAME 8080:80

After following the above instructions (port-forwarding) or using any other method you use to access Kubernetes services you should be in the Predator UI.

Writing the first load test

To do so just go to the Tests section and press the Create Test button, use the form to configure your load test.

Run it

In the above setup we requested Predator to run with 10,000 arrival rates meaning firing 10,000 scenarios each second (currently each scenario includes only one HTTP call), we also told Predator to use a parallelism level of 8 meaning to split the load between 8 pods and to run the test for 5 minutes.

Let’s check what is going on in the cluster using kubectl.

kubectl get pods | grep "predator"medium-tutorial-predator-5c78cbd96c-sn8pp                         2/2     Running   0          37mpredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510274w4   2/2     Running   0          3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725105dps9   2/2     Running   0          3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725107fmqt   2/2     Running   0          3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725108xdtf   2/2     Running   0          3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725109k6rb   2/2     Running   0          3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510c4w6g   2/2     Running   0          3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510kx96k   2/2     Running   0          3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510v5qph   2/2     Running   0          3m45s

As you may see Predator started a job that deployed 8 different pods to create a distributed load.

Inspect results

A real-time report showing latency, RPS and status codes

Predator also supports publishing the results to Prometheus or Influx by using push-gateway and viewing more detailed reports in Grafana.

Summary