Load testing gRPC services with Gatling
I always find encoding remote procedure calls as doing CRUD on resources a waste of time. With gRPC, all I have to do is to define the RPCs and data types in .proto files. Code is then automatically generated for server side to implement and client side to call. No time is wasted on
- bikeshedding where to put parameters (should this piece of information be in the path, in the query string or in the body?),
- documenting the request response schemas¹, or
- writing the same data types multiple times in different languages.
Since gRPC is quite new, I did run into a few problems². The greatest challenge came when I had to stress test my service. Unlike HTTP, there was not a sophisticated tool for me to build scenarios for testing gRPC services.
Gatling is a load and performance testing framework that supports HTTP well. There are some extensions to it that supports other protocols. “Of course I can do the same for gRPC!”, so I thought.
The schedule for the stress test was rather tight. I had only two weeks. Referring to some previous work³, in the first few days I hacked together some abstractions on top of Gatling’s core APIs to make gRPC requests. The code is ugly, of course, as I did not have time to understand the design of Gatling, but that was enough for me to get the tests running.
Learning about the performance characteristics⁴ of my system from the tests, I managed to improve the robustness quite a bit.⁵
Now that I am less busy, I have extracted my code into a library.
https://github.com/phiSgr/gatling-grpc
A Brief Introduction to Gatling
Gatling provides a domain specific language (DSL) to define test scenarios. The control flow provided by Gatling includes pausing, looping and branching. The DSL also includes “variables” in the virtual user’s state (called session attributes).
Take a look at this snippet from the Quickstart guide, the action for the virtual user to take, http("request_1").get("/"), is a Scala object.⁶
val scn = scenario("BasicSimulation")
.exec(http("request_1")
.get("/"))
.pause(5)By chaining these Action objects up with the functions in Gatling, we have a “program” that defines a user’s behaviour.
To use the session attributes easily, Gatling has a concept called “expression”, which is a value calculated/taken from the user session.
A way to construct expressions is the “Expression Language” (EL). With the magic⁷ of Scala implicits, a string "${v}" is converted to an expression that takes the value named "v" from the session attributes. EL is particularly handy when you need to do string interpolation with the session attributes. For example, to construct a JSON, using session attributes “name” and “username”:
{
"name": ${name.jsonStringify()},
"username": ${username.jsonStringify()}
}I hope I have introduced Gatling well enough that the following sharing on my library isn’t total gibberish to you. But if it is, that’s all my fault.
Now assume you have the following in your .proto file.
message HelloWorld {
string username = 1;
string name = 2;
}service GreetService {
rpc Greet (HelloWorld) returns (ChatMessage);
}
By integrating ScalaPB into the build process, along with many other things, a Scala case class (like a data class in Kotlin) HelloWorld is generated from that message type, and a method descriptor GreetServiceGrpc.METHOD_GREET is generated from rpc Greet.
To make a gRPC call, (or more accurately, to define an Action that makes a gRPC call):
grpc("my_request")
.rpc(GreetServiceGrpc.METHOD_GREET)
.payload(HelloWorld(
username = "myUserName",
name = "My name"
))In my earlier attempts, I used method references to refer to the services and rpc calls. That API is still kept for some extra flexibility⁸. But using the method descriptor gives much better looking code.
For generating requests messages, I also figured out a new way.
The payload above is static, independent of the user’s state. Let’s see how we can use the session attributes.
Previously we looked at an example of using Gatling’s Expression Language to construct a JSON, but EL is not as useful with gRPC, as the requests and responses are strongly typed protocol buffer messages. Besides, the helpful IDE enabling string interpolation⁹ whenever ${ is typed is rather annoying.
Initially I defined a function $. $("v") does the same thing as "${v}", but without needing implicit conversions. I also defined flatMap for expressions so that I can use for-comprehension to compose them.
for {
name <- $[String]("name")
username <- $[String]("username")
} yield HelloWorld(name, username)That worked. Then a few months later I used Gatling HTTP¹⁰, and used EL to generate JSON payloads.
Comparing to the EL JSON example a few paragraphs earlier, this one needs the extra meaningless variables and the type parameters that could’ve been inferred. I envied EL’s simplicity.
Then I found a way to combine lenses with expressions.
Lenses are a way to refer to a field in a struct, or a nested subfield in another struct. For an introduction to the concept, there is a tutorial in Haskell.
In the generated code, for every message type, the lens for it is also generated. We can update a message like so:
helloWorld.update(
_.name := name,
_.username := username
)With some implicit conversion, we can update the message from an expression.
helloWorld.updateExpr(
_.name :~ $("name"),
_.username :~ $("username")
)Initially my tests just look at whether the call succeeded, without any checking on the response. I wanted to design the checking API in the spirit of Gatling HTTP checks¹¹, but then I had to make some changes to help type inference.
In an HttpCheck, to extract a value from a JSON one writes.check(jsonPath("$.token") saveAs "token").
Similarly I want to write .check(extract(_.token.some) saveAs "token") in a GrpcCheck, but the compiler complains:missing parameter type for expanded function.
Type inference failed as the type information cannot “flow back up” to the function. I can provide the information like this.check(extract { c: RegisterResponse => c.token.some } saveAs "token"), but that feels too verbose.
So I provided extract in GrpcCallActionBuilder. To use it to extract a value from response and saving it to user session:
grpc("Register")
.rpc(GreetServiceGrpc.METHOD_REGISTER)
.payload(
RegisterRequest.defaultInstance.updateExpr(
_.username :~ $("username")
)
)
.extract(_.token.some)(_ saveAs "token")Expecting a failed call with a specific gRPC status code looks similar enough to the HTTP check (.check(status is 403)) :
grpc("Expect PERMISSION_DENIED")
.rpc(GreetServiceGrpc.METHOD_GREET)
.payload(HelloWorld(username = "DoesNotExist"))
.header(TokenHeaderKey)($("token"))
.check(statusCode is Status.Code.PERMISSION_DENIED)Further Work
Streaming, gRPC supports streaming in both directions. Currently my code only supports unary calls. I will try to design the streaming support similar to the WebSocket one.
Channel sharing, Gatling supports sharing of the HTTP client between virtual users, whereas in the current implementation of my library there is one ManagedChannel per user.
Throttling, it is possible to specify a goal request per second count after which the calls will be throttled, but I have not yet implemented that for gRPC calls.
EDIT: channel sharing and throttling are added in 0.3.0.
[1] You still need to inform users the semantics of the services if they are not self-evident from the RPC names and message types. But documentation on input/output format is now unnecessary as they are now part of the code.
[2] E.g., load balancers did not quite support HTTP/2, on which gRPC is based. Luckily, for my use case, load balancing at the connection level is good enough.
[3] These two projects helped a lot in understanding the APIs I had to implement.
http://callistaenterprise.se/blogg/teknik/2016/11/26/gatling-custom-protocol/
https://github.com/tamediadigital/grpc-gatling/
[4] as well as something embarrassingly fundamental, as I was a noob (still am).
[5] The Adaptive Concurrency Limits from Netflix is a lifesaver.
[6] That’s why we say the DSL is embedded in the host language Scala.
[7] In software engineering, “magic” often carries negative connotations, this instance is no exception.
[8] For example, it allows writing some Scala code to measure the time for first message to arrive in a streaming call.
[9] Sorry, that is confusing. There are two string interpolations we are talking about, the one in Scala language and the Gatling EL.
In Scala, string interpolation needs to be turned on. "v is ${v}" is just a plain string; but with an s before the string, s"v is ${v}" puts the variable v in scope into the string.
By importing io.gatling.core.Predef.stringToExpression, the former can be implicitly converted to an expression, and Gatling can put the session attribute named "v" into the string.
[10] Funny that my experience with the supported usage of Gatling comes after the unsupported one.
[11] It took me quite some time to understand the type parameters of the checks. For a few moments I sympathize with Go users who dislike generics.
