Distributed load testing with Kubernetes and Predator
In this tutorial, we are going to use an open-source performance platform designed for APIs crafted at ZooZ (a PayU company) called ‘Predator’.
Predator is a distributed performance testing platform that uses Kubernetes abilities to schedule a distributed load on your APIs which is aggregated to one report in real-time.
Deployment
While Predator supports more platforms like Docker engine and DC/OS we are going to focus on Kubernetes.
The simplest way to deploy Predator is by using the official helm chart.
Predator uses persistence to save all tests, reports and scheduled runs.
You can take advantage of Kubernetes Persistent storage or connect it to other data sources like Cassandra, PostgreSQL, MySql, and SQL Server.
For simplicity let’s go with persistence storage.
helm repo add zooz https://zooz.github.io/helm/helm install medium-tutorial zooz/predator --set persistence.enabled=true
If everything went well you should get the following output:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods -l "app=medium-tutorial-predator,release=medium-tutorial" -o jsonpath="{.items[0].metadata.name}")echo "Visit http://127.0.0.1:8080 to use your application"kubectl port-forward $POD_NAME 8080:80
After following the above instructions (port-forwarding) or using any other method you use to access Kubernetes services you should be in the Predator UI.
Writing the first load test
We are going to create our first API test using the Create test form.
While Predator supports composing complex load tests with weights between multiple scenarios, custom javascript injection, before section, capturing values from the responses and more (you can find all about this in the docs), we are going to write a one-step test that load tests a bin details service (a service which receives the first 6 digits of a credit card and returns info about the vendor, country, etc..)
To do so just go to the Tests section and press the Create Test button, use the form to configure your load test.
Run it
This is when the fun begins, Predator can use your cluster capacity to create tremendous load and aggregate all of it to one report in real-time by clicking one button.
To run a test just press the Run button under the Run Test column and configure your parameters.
In the above setup we requested Predator to run with 10,000 arrival rates meaning firing 10,000 scenarios each second (currently each scenario includes only one HTTP call), we also told Predator to use a parallelism level of 8 meaning to split the load between 8 pods and to run the test for 5 minutes.
Let’s check what is going on in the cluster using kubectl.
➜ kubectl get pods | grep "predator"medium-tutorial-predator-5c78cbd96c-sn8pp 2/2 Running 0 37mpredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510274w4 2/2 Running 0 3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725105dps9 2/2 Running 0 3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725107fmqt 2/2 Running 0 3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725108xdtf 2/2 Running 0 3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-1579283725109k6rb 2/2 Running 0 3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510c4w6g 2/2 Running 0 3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510kx96k 2/2 Running 0 3m45spredator.cfe37ec7-59c2-498c-bc5e-acb8cc07333f-157928372510v5qph 2/2 Running 0 3m45s
As you may see Predator started a job that deployed 8 different pods to create a distributed load.
Inspect results
all the running pods are reporting to Predator about their results and predator aggregates everything in real-time to one visual report.
Predator also supports publishing the results to Prometheus or Influx by using push-gateway and viewing more detailed reports in Grafana.
Summary
In this short tutorial, we went over the basics of Predator.
you can find much more information regarding its functionality and features that are not covered in the docs.
