gRPC Load Test by Locust with k8s

Published in

Arbeon

6 min readJan 10, 2023

Hello! It’s nice to meet you.
We’re Donis and Riley, who oversee the server development in Arbeon!

Before we get into the first-ever Tech article by us, the Server team, we would like to introduce you briefly to what we do in the company.

As you can guess from the name “Server,” the word is a compound word of “serve” (action) and “-er” (doer). That is, it means “the person who provides the service.” We, the Server team, create a quick, stable, and expandable environment in advance, predict possible risks and errors in the service, and minimize them in case those issues occur. The stable operation of the Arbeon service is in the hands of the Server team!

That was quite a long introduction, wasn’t it? Wait no more.

Let’s get started with our first article right away!

gRPC Load Test by Locust with k8s

Searching for the faster and more stable communication

At the moment, Arbeon’s server application is developed using the go language and composed of the microservice that uses gRPC communication protocol. The Rest API communication that uses the Nest.js framework was previously used to communicate between the clients and the server. But gRPC was selected as the final choice as it ensures faster and more stable communication. As a result, we were able to achieve speed three times faster than the Rest API.

So now, let’s look at how we can load test using the gRPC!

Load Test

Load test refers to the test that checks the system function and predicts the possible issues by simulating several users and accessing a certain program or app at the same time. The test allows us to increase the load to the set value or threshold to compare the functions of various systems, or accurately measure the function of a single system. Moreover, it is done to measure how well it does in running the actual system.

Load Test Checklist

How much user access can it hold?
Are there any issues with the expandability when the acceptance range is exceeded?
Is the current infrastructure enough?
Can the maximum DB and network workload, which are related to the server application, be determined and expanded in the future?
Are there any errors and issues like memory leak?

Looking into the Tools for the Load Test

Commonly, in most services, things may go smoothly in the developing environment until the service is run in the actual environment. That’s when unexpected issues and exemptions start to occur. And so, it’s nearly impossible to run a test that predicts every single error and exemption during the developing and coding stage.

For this reason, we looked into several tools to test out the gRPC API we use under conditions very similar to the actual service environment. Comparing each of them, we came to choose Locust because it can be easily tested in a single system based on the test case, particularly the Kubernetes (k8s) environment.

Creating the Load Test Environment

First, the Locust environment must be set to write the test case. Since it should be written in Python, the program and gRPC dependencies library must be installed.

Installing the Python gRPC Dependencies Library

# Python upgrade python >= 3.6
> python3 -m pip install --upgrade pip# install grpcio 
> python3 -m pip install grpcio# install grpcio-tools
> python3 -m pip install grpcio-tools

Compiling the Protobuf for gRPC Test Object

# compile proto filepython3 -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. proto/v1/user/user.proto
python3 -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. proto/v1/feed/feed.proto

* If you want to find more about the tutorial, click the link on the right. ▶ Python gRPC Basics tutorial

Writing the Test Case File for gRPC

import sys
import grpc
import inspect
import time
import gevent# Libs
from locust.contrib.fasthttp import FastHttpUser
from locust import task, events, constant, between
from locust.runners import STATE_STOPPING, STATE_STOPPED, STATE_CLEANUP, WorkerRunner
from proto.v1.user import user_pb2
from proto.v1.user import user_pb2_grpc
from proto.v1.feed import feed_pb2
from proto.v1.feed import feed_pb2_grpcdef stopwatch(func):    def wrapper(*args, **kwargs):
        # import function name
        previous_frame = inspect.currentframe().f_back
        _, _, task_name, _, _ = inspect.getframeinfo(previous_frame)        start = time.time()
        result = None
        try:
            result = func(*args, **kwargs)
        except Exception as e:
            total = int((time.time() - start) * 1000)
            events.request_failure.fire(request_type="TYPE",
                                        name=task_name,
                                        response_time=total,
                                        response_length=0,
                                        exception=e)
        else:
            total = int((time.time() - start) * 1000)
            events.request_success.fire(request_type="TYPE",
                                        name=task_name,
                                        response_time=total,
                                        response_length=0)
        return result    return wrapper
class GRPCMyLocust(FastHttpUser):
    host = 'https://grpc.bla-bla-비밀.com:443'
    wait_time = between(1, 5)
    # we separately added the pem file because we use the TLS for gRPC connection.
    creds = grpc.ssl_channel_credentials(open('./cert.pem', 'rb').read())
    access_token = ""
    try:
        replace_host = host.replace('https://', '')
        channel = grpc.secure_channel(replace_host, creds)
        stub = user_pb2_grpc.UserStub(channel)
        response = stub.DefaultLogin(user_pb2.DefaultLoginRequest(user_id='donis', password='1234'))
        # save after issuing the token
        access_token = response.data.access_token
    except (KeyboardInterrupt, SystemExit):
        sys.exit(0)    def on_start(self):
        # write here if something needs to be executed before doing the @task.
        pass    def on_stop(self):
        # write here if you something needs to be executed after the @task is finished.
        pass    @task
    @stopwatch
    def grpc_client_task(self):
        try:
            # there is no 'http://' or 'https://' protocol name in the front for gRPC connection, so replace them.
            replace_host = self.host.replace('https://', '')
            channel = grpc.secure_channel(replace_host, self.creds)
            feed_stub = feed_pb2_grpc.FeedServiceStub(channel)            metadata = (('authorization', self.access_token),)
            request = feed_pb2.FeedTimelineRequest(user_id='donis', sort_type='RECENT', last_feed_id='0')
            response = feed_stub.GetFeedTimeline(request=request, metadata=metadata)
            # print(response)
        except (KeyboardInterrupt, SystemExit):
            sys.exit(0)
# quit locust if it goes over fail ratio
def checker(environment):
    while not environment.runner.state in [STATE_STOPPING, STATE_STOPPED, STATE_CLEANUP]:
        time.sleep(1)
        if environment.runner.stats.total.fail_ratio > 0.2:
            print(f"fail ratio was {environment.runner.stats.total.fail_ratio}, quitting")
            environment.runner.quit()
            return
# initial setting
@events.init.add_listener
def on_locust_init(environment, **_kwargs):
    if not isinstance(environment.runner, WorkerRunner):
        gevent.spawn(checker, environment)

Write locust.py @Task file(gRPC)

Conducting the Load Distribution Test in k8s Environment

The services of the Arbeon server are composed of Container, which is managed using k8s. In the case of Locust, it is packaged with Helm for an easy deployment in k8s, and so the load test in the k8s environment is simple.

Building the Docker Image for the Locust Test Case

# Dockerfile
FROM locustio/locust:2.10.1
RUN pip3 install grpcio
RUN pip3 install grpcio-toolsCOPY . /home/locust/.

It must be built according to the infrastructure architecture that will be deployed during the Docket Build. Since we are using MacBook M1, we built it as below:

docker docker buildx build --platform linux/amd64 -t locust-grpc:0.0.1 .

Registering the Docker Image in the Repository

Push the built images to AWS ECR or Dockerhub.

docker push public.ecr.aws/blahblah/locust-grpc:0.0.1

Using the Helm Chart

Once the cluster installment environment of the load test is created using the Helm chart, the locust cluster can be easily installed or deleted.
Add the deliveryhero/locust available in Delivery Hero in the Helm repository and update.

helm repo add deliveryhero ""
helm repo update

Write the setting file values.yaml and Install the Locust Helm Chart

# values.yaml
loadtest:
name: grpc-loadtest
locust_locustfile: ../../home/locust/locustfile.py
master:
image: public.ecr.aws/blahblah/locust-grpc:0.0.1
worker:
image: public.ecr.aws/blahblah/locust-grpc:0.0.1
hpa:
enabled: true
minReplicas: 5helm upgrade --install locust deliveryhero/locust -f values.yaml

deliveryhero/locust installed!

locust access screen

In Locust, the address starting with “http://” or “https://” must be inserted when entering the host (For this reason, we inserted the logic to replace the host address when we wrote the code for the locust.py test).

Conclusion: The Result of the Load Test

The Test Result of Locust Using MacBook