We get asked about this all the time. In general, gRPC and RSocket attempt to solve different problems. gRPC is an RPC framework using HTTP/2. RSocket is a message passing network layer that is lower level. So, a developer would use RSocket directly for low-level interaction and has the option to use RSocket-RPC as an easy to use RPC layer that sits on top of RSocket.

Now, let’s look more specifically at differences between gRPC and RSocket.

OSI Layer

gRPC and RSocket are at different layers in the stack. gRPC is at OSI layer 7 — an RPC layer built on top of HTTP/2. RSocket is an OSI layer 5/6 that models Reactive Streams semantics over a network. Reactive Streams provides a way to model asynchronous streams with back-pressure. At a 10,000-foot view, you would use RSocket to build something like gRPC — RSocket-RPC for example — which would be at the OSI (application) layer 7.

Protocol

gRPC is not a protocol in the traditional sense. It is comprised of HTTP/2 headers, generated code, and conventions in protobuf. There is not enough information in what is being transmitted across the network to determine what is happening. For instance, gRPC can have a unary call make a call to a stream if there is mismatch in the protobuf. gRPC design is analogous to Web Services.

RSocket is a layer 5/6 binary protocol with a formal definition. You do not need to generate code to determine what is happening with RSocket — you only need to understand the protocol.

Payloads

gRPC is opinionated about the payload you are sending — for all intents and purposes it’s protobuf.

RSocket is not opinionated and lets you send opaque bytes.

Advantages

Sending bytes is more flexible as you do not have to define a protobuf for sending data. RSocket could be used to transport database queries just as easily as transfer a picture.

Network Transport

gRPC is designed to work with HTTP/2 semantics. If you want to send it over a different transport you must mimic HTTP/2 semantics. Across the network it is effectively bound to HTTP/2 and TCP.

RSocket only needs a duplex connection — something that can send and receive bytes. This can be TCP, a File, WebSockets, etc. It is very easy in RSocket to receive a call from a browser via WebSockets and then make a call to a server using TCP. The interaction would feel transparent and identical to a developer.

Browser Support

gRPC does not work in a web browser. Support in a web browser requires additional code to be generated and deployed.

RSocket works in a web browser via Websockets. No new code is needed — all that’s needed is to start an RSocket server that accepts WebSocket connections.

Client / Server Interaction

gRPC has a traditional client/server model because it is based on HTTP/2 and RPC semantics. In a gRPC, a client connects to the server, but the server cannot make calls to a client. Additionally, because gRPC is an RPC layer you can only make calls defined in your protobuf file.

RSocket does not have a client server interaction in the traditional HTTP sense. In RSocket, once a client connects to a server, they both can be a requester and responder. This means that after they are connected, RSocket is fully duplex and server’s requester can initiate calls to a client’s requester — i.e. a server can make calls to a web browser once connected.

Flow Control

gRPC’s flow control is byte-based because it is ultimately HTTP/2 flow control. It does not have a concept of application flow control because it does not send messages between participants with this information. HTTP/2 flow control has limited value because TCP also provides byte level flow control.

RSocket’s flow control is application level flow control. This is governed by how many messages a can a recipient handle. RSocket has a REQUEST_N frame that is shared between the participants requesting a number of messages. Once those messages are handled, it requests more messages. This is independent of the underlying transports byte-level flow control. This means if a service slows down, it can slow down the number messages before network buffers are full. This information is propagated between services.

Cancellations

In gRPC, cancellations generally only come from a client to a server. The server is only supposed to send a cancel if a sequence is incomplete.

RSocket cancellations are sent between requester and responder. It does not matter which side is a client or a server. Additionally, because of reactive stream semantics they are automatically propagated between services.

Load Balancing

gRPC is limited to HTTP/2 load balancing. You cannot load balance unary gRPC calls differently than Streaming calls. This means that a short-lived request/reply call is load-balanced the same as a long-running stream.

RSocket load balancing can happen at a transport level and the application level. This is because there is complete information about what is happening encapsulated in the message. You can create a load balancer that treats request/replies differently than request/streams.

Spatial Coupling

gRPC is coupled to the URL and service that you are sending the request to. There must be a recipient on the other end to receive the call because the message is sent directly to them. If the recipient doesn’t exist, the call will fail. This is still true if you place a proxy between the client and server. The proxy then must exist. If it does not exist, the request will fail. If there isn’t anything for the proxy to call then its requests will fail as well.

RSocket is messaging passing based — it is not opinionated about where its frames are sent. The recipient does not need to exist on the other side when the message is sent because the message is sent via a stream — the frames in the stream merely need to handled. Also, as the frames in the streams are handled according to the RSocket protocol, it doesn’t matter where they are handled or who handles them.

This means with RSocket it is possible to model interactions like multi-cast and broadcast.

Interaction Models

gRPC sends information over the network as an HTTP/2 stream. This means for request/reply and fire/forget interactions, more frames must be sent over the network. Also, with gRPC, information about streaming is encapsulated in application code. This means it is only possible to tell the difference between a GET and POST by looking at application code.

RSocket has a concept of interactions built into the protocol. This is similar in concept (but not function) to HTTP methods. The interaction models are request-reply (send one/receive one), fire-and-forgot (send one), request stream (send one/receive many), and request channel (send many/receive many). These are included in frames in the protocol and allow for implementations to optimize for the different interaction models. Streams send over a single connection.

Developer APIs

The gRPC API is limited to the code generated by gRPC, and you must use an RPC-style interaction.

RSocket is more flexible. If you like RPC-style semantics, there is RSocket-RPC. If don’t like that, you can use Spring Messaging. And, if you are comfortable with lower-level application programming, you can use RSocket directly.

Temporal Coupling

Both gRPC and RSocket provided non-blocking APIs.

Keep Alive and Health Checks

gRPC has both keep alive and a health check. The health check is special protobuf that can be implemented.

RSocket just uses a keep alive. If the keep alive fails, the connection is terminated.

Resumption

RSocket supports connection resumption. If a connection fails, RSocket can re-connect automatically to give the impression of a stable connection.

RSocket Java Specific Differences

Zero Copy

RSocket Java provides an API that lets developers manage the Netty ByteBuf lifecycle allowing for zero copy.

Automatic Request Batching

RSocket Java takes advantage of Reactive Stream semantics to batch requests over the network. Using this information, RSocket can determine to flush data when network buffers are full, or if it needs to flush based on the information about the stream. The result is that RSocket will send the optimal amount of data over the network based on what it observes without having to be tuned by the user.

Flyweight-Based Frames

RSocket Java uses the flyweight pattern to encode and decode frames. These flyweights do not allocate any data. Instead, they act as a lens over a ByteBuf to read without the overhead of decoding the frame.

RSocket Programming APIs

RSocket is an OSI layer 5/6 protocol, and while it can be used directly depending on the use-case, it is much easier to develop applications with a layer 7 protocol. Currently there is RPC, IPC, and soon to added GraphQL support. The most mature right now is RSocket RPC, which is a protobuf auto-marshalling layer that sits on top of RSocket to make message passing convenient. At its core it is not RPC — it is a message-passing layer that generates boilerplate methods and auto-marshals messages. It does this with a protobuf compiler. Unlike gRPC, which creates HTTP/2 messages with information in headers, RSocket RPC generates binary encoded frames. The frames have been standardized into the RSocket extension frames.

RSocket RPC shares RSocket’s concept of a requester and responder. This means that a client can connect to a server, and that server can make calls to a service the client is hosting.

All information about an interaction is encapsulated in the frames sent between requester and responder. You do not need generated code to determine how to handle the message. This means that you do not need generated code to understand RSocket RPC calls. If you do not like RPC-generated code, you can instead use IPC-style code to handle a message. The IPC code also allows you to specify what serializers to use and allows you to use send methods other than protobuf.

GraphQL support is being added to use the IPC-style code. This will support query, mutation, and subscription using GraphQL.

Differences between gRPC and RSocket was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.

We get asked about this all the time. In general, gRPC and RSocket attempt to solve different problems. gRPC is an RPC framework using HTTP/2. RSocket is a message passing network layer that is lower level. So, a developer would use RSocket directly for low-level interaction and has the option to use RSocket-RPC as an easy to use RPC layer that sits on top of RSocket.

Now, let’s look more specifically at differences between gRPC and RSocket.

OSI Layer

gRPC and RSocket are at different layers in the stack. gRPC is at OSI layer 7 — an RPC layer built on top of HTTP/2. RSocket is an OSI layer 5/6 that models Reactive Streams semantics over a network. Reactive Streams provides a way to model asynchronous streams with back-pressure. At a 10,000-foot view, you would use RSocket to build something like gRPC — RSocket-RPC for example — which would be at the OSI (application) layer 7.

Protocol

gRPC is not a protocol in the traditional sense. It is comprised of HTTP/2 headers, generated code, and conventions in protobuf. There is not enough information in what is being transmitted across the network to determine what is happening. For instance, gRPC can have a unary call make a call to a stream if there is mismatch in the protobuf. gRPC design is analogous to Web Services.

RSocket is a layer 5/6 binary protocol with a formal definition. You do not need to generate code to determine what is happening with RSocket — you only need to understand the protocol.

Payloads

gRPC is opinionated about the payload you are sending — for all intents and purposes it’s protobuf.

RSocket is not opinionated and lets you send opaque bytes.

Advantages

Sending bytes is more flexible as you do not have to define a protobuf for sending data. RSocket could be used to transport database queries just as easily as transfer a picture.

Network Transport

gRPC is designed to work with HTTP/2 semantics. If you want to send it over a different transport you must mimic HTTP/2 semantics. Across the network it is effectively bound to HTTP/2 and TCP.

RSocket only needs a duplex connection — something that can send and receive bytes. This can be TCP, a File, WebSockets, etc. It is very easy in RSocket to receive a call from a browser via WebSockets and then make a call to a server using TCP. The interaction would feel transparent and identical to a developer.

Browser Support

gRPC does not work in a web browser. Support in a web browser requires additional code to be generated and deployed.

RSocket works in a web browser via Websockets. No new code is needed — all that’s needed is to start an RSocket server that accepts WebSocket connections.

Client / Server Interaction

gRPC has a traditional client/server model because it is based on HTTP/2 and RPC semantics. In a gRPC, a client connects to the server, but the server cannot make calls to a client. Additionally, because gRPC is an RPC layer you can only make calls defined in your protobuf file.

RSocket does not have a client server interaction in the traditional HTTP sense. In RSocket, once a client connects to a server, they both can be a requester and responder. This means that after they are connected, RSocket is fully duplex and server’s requester can initiate calls to a client’s requester — i.e. a server can make calls to a web browser once connected.

Flow Control

gRPC’s flow control is byte-based because it is ultimately HTTP/2 flow control. It does not have a concept of application flow control because it does not send messages between participants with this information. HTTP/2 flow control has limited value because TCP also provides byte level flow control.

RSocket’s flow control is application level flow control. This is governed by how many messages a can a recipient handle. RSocket has a REQUEST_N frame that is shared between the participants requesting a number of messages. Once those messages are handled, it requests more messages. This is independent of the underlying transports byte-level flow control. This means if a service slows down, it can slow down the number messages before network buffers are full. This information is propagated between services.

Cancellations

In gRPC, cancellations generally only come from a client to a server. The server is only supposed to send a cancel if a sequence is incomplete.

RSocket cancellations are sent between requester and responder. It does not matter which side is a client or a server. Additionally, because of reactive stream semantics they are automatically propagated between services.

Load Balancing

gRPC is limited to HTTP/2 load balancing. You cannot load balance unary gRPC calls differently than Streaming calls. This means that a short-lived request/reply call is load-balanced the same as a long-running stream.

RSocket load balancing can happen at a transport level and the application level. This is because there is complete information about what is happening encapsulated in the message. You can create a load balancer that treats request/replies differently than request/streams.

Spatial Coupling

gRPC is coupled to the URL and service that you are sending the request to. There must be a recipient on the other end to receive the call because the message is sent directly to them. If the recipient doesn’t exist, the call will fail. This is still true if you place a proxy between the client and server. The proxy then must exist. If it does not exist, the request will fail. If there isn’t anything for the proxy to call then its requests will fail as well.

RSocket is messaging passing based — it is not opinionated about where its frames are sent. The recipient does not need to exist on the other side when the message is sent because the message is sent via a stream — the frames in the stream merely need to handled. Also, as the frames in the streams are handled according to the RSocket protocol, it doesn’t matter where they are handled or who handles them.

This means with RSocket it is possible to model interactions like multi-cast and broadcast.

Interaction Models

gRPC sends information over the network as an HTTP/2 stream. This means for request/reply and fire/forget interactions, more frames must be sent over the network. Also, with gRPC, information about streaming is encapsulated in application code. This means it is only possible to tell the difference between a GET and POST by looking at application code.

RSocket has a concept of interactions built into the protocol. This is similar in concept (but not function) to HTTP methods. The interaction models are request-reply (send one/receive one), fire-and-forgot (send one), request stream (send one/receive many), and request channel (send many/receive many). These are included in frames in the protocol and allow for implementations to optimize for the different interaction models. Streams send over a single connection.

Developer APIs

The gRPC API is limited to the code generated by gRPC, and you must use an RPC-style interaction.

RSocket is more flexible. If you like RPC-style semantics, there is RSocket-RPC. If don’t like that, you can use Spring Messaging. And, if you are comfortable with lower-level application programming, you can use RSocket directly.

Temporal Coupling

Both gRPC and RSocket provided non-blocking APIs.

Keep Alive and Health Checks

gRPC has both keep alive and a health check. The health check is special protobuf that can be implemented.

RSocket just uses a keep alive. If the keep alive fails, the connection is terminated.

Resumption

RSocket supports connection resumption. If a connection fails, RSocket can re-connect automatically to give the impression of a stable connection.

RSocket Java Specific Differences

Zero Copy

RSocket Java provides an API that lets developers manage the Netty ByteBuf lifecycle allowing for zero copy.

Automatic Request Batching

RSocket Java takes advantage of Reactive Stream semantics to batch requests over the network. Using this information, RSocket can determine to flush data when network buffers are full, or if it needs to flush based on the information about the stream. The result is that RSocket will send the optimal amount of data over the network based on what it observes without having to be tuned by the user.

Flyweight-Based Frames

RSocket Java uses the flyweight pattern to encode and decode frames. These flyweights do not allocate any data. Instead, they act as a lens over a ByteBuf to read without the overhead of decoding the frame.

RSocket Programming APIs

RSocket is an OSI layer 5/6 protocol, and while it can be used directly depending on the use-case, it is much easier to develop applications with a layer 7 protocol. Currently there is RPC, IPC, and soon to added GraphQL support. The most mature right now is RSocket RPC, which is a protobuf auto-marshalling layer that sits on top of RSocket to make message passing convenient. At its core it is not RPC — it is a message-passing layer that generates boilerplate methods and auto-marshals messages. It does this with a protobuf compiler. Unlike gRPC, which creates HTTP/2 messages with information in headers, RSocket RPC generates binary encoded frames. The frames have been standardized into the RSocket extension frames.

RSocket RPC shares RSocket’s concept of a requester and responder. This means that a client can connect to a server, and that server can make calls to a service the client is hosting.

All information about an interaction is encapsulated in the frames sent between requester and responder. You do not need generated code to determine how to handle the message. This means that you do not need generated code to understand RSocket RPC calls. If you do not like RPC-generated code, you can instead use IPC-style code to handle a message. The IPC code also allows you to specify what serializers to use and allows you to use send methods other than protobuf.

GraphQL support is being added to use the IPC-style code. This will support query, mutation, and subscription using GraphQL.

There’s a plethora of advice out there about what you “need” to be a microservice. You need AWS and Docker and Kubernetes and use 128 megs of ram. Otherwise, you’re not doing microservices. The truth of the matter is that this list has nothing to do with microservices. Absolutely nothing.

Microservices are an architectural decision to align your services with a company’s business units. It is understandable that there is confusion between deployment, packaging, scheduling, and resources utilization and microservices themselves. The motivation behind these concepts is not the same reason you move to microservices — even if they are often used in concert. Let’s take a quick look and see if we can make the situation less confusing.

Not about Deployment

Moving to the cloud is motivated by two main desires — shifting your business from capital expenditures to operating expenditures and gaining new compute capacity without having to rack and stack servers. Deploying your monolith to the cloud doesn’t magically transform it into a microservice. All you’ve done is deploy a monolith to the cloud.

Not about Packaging

What about Docker? It’s definitely for microservices. Alas, no, it is not. A container allows you to put libraries and applications into a reusable package. There are no rules about what can run in this package. It can be a tiny one API app or a 10 gig monster with a thousand APIs. Containers offer a convenient package to deploy your applications in without having to worry about library conflicts etc. Packing your monolith in a container doesn’t magically transform it into a microservice. All you’ve done is put a monolith in Docker.

Not about Scheduling

Kubernetes is not an application server. It schedules Docker containers — and runs whatever is contained within. It has some other features to help make the scheduled containers available, but its primary purpose is to utilize computing resources efficiently. If anything, Kubernetes is replacement for something like vSphere. Scheduling your application using Kubernetes doesn’t magically transform it into a microservice. All you’ve done is schedule a monolith in Kubernetes.

Not about Resources

Everyone talks about how small microservices are — so it’s the size of service, right? Resource usage doesn’t determine if it’s a microservice or not. There are very resource-light monoliths and incredibly resource-hungry microservices. It’s actually possible to make your monolith more resource-hungry in aggregate when you switch to a microservice architecture because of all the serialization that you end up doing (a topic for another day). Making your monolith resource-efficient doesn’t magically make it a microservice…

It’s about the Code

Microservices are an application implementation detail. It doesn’t matter where you deploy it, how you package it, how you schedule it, or how resource intense it is. If you want to use microservices, architect your application that way. It’s only a microservice if it is coded that way.

Four Myths about Microservices was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.

Spring WebFlux is a great way to create a non-blocking REST application. One issue that you run into when start working with WebFlux is JDBC. JDBC is blocking. New school databases like Cassandra or Couchbase have non-blocking drivers. In Couchbase’s case its driver uses RXJava. NoSQL databases are great, but you can’t always use one for various reasons. There is some effort going into creating asynchronous drivers for databases, as well as Oracle’s effort to create ADBA. Unfortunately, these are early days, and if you want to talk to a SQL database on the JVM you’re stuck with a blocking driver. This is a problem when you’re building a non-blocking application with Spring WebFlux . Luckily reactor-core and rxjava2 both implement ReactiveStreams.

This means that you can use David Moten’s rxjava2-jdbc library. It takes a normal JDBC driver and allows you to interact with it in a way that won’t block your application. It does this by scheduling potentially blocking work on different threads. Additionally, it has DSL that lets you model your SQL statements and results into streams. This makes integrating into your reactive application a snap.

Rxjava2-jdbc also provides non-blocking connection pooling. Normally in a blocking application threads are blocked until a connection is available to make a query. This is a problem for a non-blocking application as you will quickly exhaust all database connection threads and turn your wonderful non-blocking application into a clunky blocking application. In rxjava2-jdbc its connection pool returns a Single that is controlled by the connection pool. This means that instead of assigning threads to each query, queries subscribe to the connection pool and receive a connection when one is available — no blocked threads!

Nothing is better than an example, so I created one that uses Spring WebFlux, rxjava2-jdbc, and H2. The example is a simple REST application that uses an embedded H2 database. It can be found here. The example has two objects. The first is an Employee object which contains information about the employee including: first name, last name, and department id. The second object is the Department object, which contains information about different departments. The application has restful endpoints for basic create, read, delete and list operations for the Employee and list operations for Department.

Running the Example

Running the example is simple and can be done with following command:

./gradlew clean run

This command will configure the database and start the WebFlux application using Spring Boot. The example doesn’t need a servlet container to run, and instead uses Netty to serve traffic. If you want to test and see if the example is up and running you can run the following curl command:

curl localhost:8080/employees

You should see a JSON employee list if it’s running properly.

Routing Requests

Let’s take look inside the application and see what it’s doing. The first thing to take a look at is the EmployeeRouter class. This class routes incoming HTTP requests to the appropriate handler based on URL and HTTP method.

In Spring WebFlux because it’s non-blocking these handlers must return a Mono<T>.

Handling Requests

The next class to take a look at is the class that is handling the requests, EmployeeHandler.

This is the class that is responsible for handling incoming requests. Each handle method returns a Mono<ServerResponse>. The ServerResponse object has a builder that is used to create the response, and has methods that take Fluxes and Monos for responses. The ServerResponse object will also automatically serialize your results to JSON. Additionally, if you return a stream of responses, WebFlux will automatically turn the response into a JSON list. You can can see this in the getAllEmployees and getAllDepoartment methods. A method to look closer at is the deleteEmployee method. The deleteEmployee method is a little different then the other methods. You’ll notice that most methods use the body method to create a response. The deleteEmployee doesn’t have a body so it uses the build method instead which takes a Mono<Void>.

Interacting with the Database

To create our responses we need to get the information from our database. The object that takes care of that is the the EmployeeRepository which is being injected into the handler. This is normally where things start to fall apart because you’re accessing a blocking database. In our case though we’re using rxjava2-jdbc and it’s no longer an issue. While this does turn a JDBC driver into a non-blocking driver and deals with the difficulties of scheduling potentially blocking work off the event loop, this isn’t as efficient as a purely non-blocking approach. However, it is sufficient for all but the highest throughput applications. Lets take a look at the EmployeeRepository class:

The first thing to look at here is the constructor. The main class in rxjava2-jdbc is the Database class. This is what you use to interact with the database and run queries. Before you can do that you need to create a connection to the database. For our example, we’re creating a connection to an H2 database. Many databases allow you to pass in user credentials and other information in there connection string. If they do then you can use just create a database like so:

Database db = Database.from(“<connection_string”>, <pool_size>);

However, H2 doesn’t allow you to do this, so you need to create a Connection object in order to pass in credentials. You then have to use the Pools builder to create a pool. Once you’ve created the your pool you can create the database object.

The great thing about rxjava2-jdbc is that any JDBC driver can used. You can switch any standard JDBC database like Oracle, SQL Server, DB2, etc.

I’m not going to go into too much detail about how rxjava2-jdbc works here because David Moten’s github page has great documentation.

I will point out a few things though about using Spring WebFlux and rxjava2-jdbc together. The good news is they both implement ReactiveStreams so it’s transparent for them to interoperate. The way to do this with reactor-core is to use the Flux.from or Mono.from methods. These can take any publisher. It is important to note that in the Mono version it can only emit one item or it will throw an exception. Another thing to take a look at is the deleteEmployee method. This method returns a Mono<Void> which is a special convention in reactor-core. This roughly equivalent to a Completable in rxjava2. To mimic this behavior in reactor-core, call the then method on a Flux or a Mono. This will ignore all elements that are emitted from the stream, and either complete or emit an exception.

Write some Code

To really get a feel for how this works, the next step is to write some code. One idea I would suggest are to port the sample app to the database of your choice. Another idea is to create endpoints for creating a new department and deleting one. You can use the createNewEmployee and deleteEmployee methods as references.

Conclusion

The reality is that many applications today use SQL databases, and reactive application developers did not have an easy way to access them without turning their non-blocking application into a blocking application. Hopefully in the future there will be great solutions around truly non-blocking JDBC drivers. Until that time, I hope you can see with this blog post, that while not perfect, there is an effective solution.

Spring WebFlux and rxjava2-jdbc was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.

The first example is a baseline — you want to run your code on the currently executing thread. It’s a pretty simple scenario, shown first in JDK8 and then in Reactor.

Here’s the reactor example:

Nothing special to see here — just iterate from 0 to 9 and print a random number. The next example is more complicated. What if you want to print the numbers on a different thread? Here’s an example using JDK8 and ForkJoinPool:

Now here’s the example using reactor:

The import thing to note here is the subscribeOn operating. In this example, it’s telling reactor to subscribe on a worker provided by the parallel scheduler. The blockLast operator is called instead of subscribe because the code is running in a different thread we must wait until it’s completed or the JVM would exit.

Another thing to note is that the subscriber of the publisher can choose to run it on different threads independent of the publisher’s creator.

In this example the ‘observable’ object was returned by a method. The subscriber could either run on the current thread or do what the example did and run it using a scheduler on potentially another thread.

This next example shows how to run each random number generation in a different thread. Here’s what it looks like using JDK8’s ForkJoinPool:

This time we had to add the ‘ForkJoinTask’ objects a list and then block on them to make sure each task was executed.

Here is the equivalent example using reactor-core:

This example uses a flatMap to map and merge the results of another Publisher together. The publishers created inside the flatMap — in this case a Mono that prints a message — is told to subscribe using a scheduler. This causes each of these different Monos to run on a different thread. The main publisher is then blocked until everything is finished. What’s interesting about this is that the Monos could be anything — a synchronous Mono that prints a message, a runnable that prints a message on a different thread, or an off-box network call. They are all treated the same.

The final example does work on two different threads. The first thread generates a random number, prints it and then makes it available to another thread. The second thread then adds up the random incoming numbers printing an intermediate total, and then a final total after it gets 10 random numbers. Here’s what the example looks like in JDK8 using the ForkJoinPool:

Things are starting to get trickier here. We must use a latch and count how many receive to make sure that we have ten items. Also, we needed us some kind of thread-safe queue to send data between the threads.

Here’s what the example looks like with reactor-core:

Before we get to the publishOn operators let’s look at the map, and reduce operators. The map operator takes an integer generated by range, and then returns the random number that is generated. This is then passed to the reduce operator. In this case we are just adding two numbers, and it will add all the different random number as they are generated. At the end we call block to wait until everything is done.

This example uses the publishOn method. The first publishOn method is before the map operator. This makes the map operators happen on another thread. There is another publishOn right after the map operator and right before a reduce operator. This will cause all the reduce operations to also happen on a different thread. There isn’t another publishOn before the doOnNext, so it happens on the same thread of the reduce operator.

It is in more complex examples like this that Reactive Streams schedulers really shine. Rather than having to spend time reasoning about complex and potentially buggy multithreaded code using Java’s concurrency utilities, having understood the scheduleOn and publishOn operators it is a simple matter to introduce parallelism where is it needed in a way that it both concise and easy to reason about.

Scheduling with Rx and Reactor was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.

This first example looks through a list of numbers, and looks for even and odd numbers. The conditional logic is in the for loop. Here’s what the code looks like when it’s imperative:

The above example is synchronous. We can do the same thing with reactor-core using the map operator. In this example the conditional logic We can map over the results and returns a string that indicates if they are even or odd.

This works well when the code is synchronous, but Rx-style programming really shines with asynchronous code. This time instead of using the map operator, we’ll use flatMap. The flatMap operator is the combination of the map and merge operator. First it applies a function that returns a Publisher — this the map part, and then it takes the results and merges them back into the stream. Here’s the first example for above but now we’re calling an asynchronous method.

The check method now returns a Mono<String> which can be asynchronous. The logic may be executed in another thread, or even depend on the result of a network call.

Let’s say this next time you want to count the number of odd and even numbers. To start here’s an example using some imperative code:

We could also use an external variable with reactor as well but there is a better way to do this. We can use the groupBy operator. The groupBy operator lets you split a single stream into multiple different streams based on the data you select. In our case, we are going to split the stream into two streams: odd and even. Then we can count the number for each stream individual. This example will use the flatMap operator to combine the results of the separated streams back together.

This example, while more complex, now completely abstracts away asynchrony. This example changes the check method to return a publisher that subscribes on another thread. None of the logic changes though!

Conditional logic in RX style programming is a big hurdle for new users. If you’re coming from an imperative background there is a learning curve. Once you get the hang of it though, you can create powerful but simple applications that would be almost to difficult the old way.

Conditional Logic and RX was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.

The first example is a baseline—you want to run your code on the currently executing thread. It’s a pretty simple scenario, shown first in JDK8 and then in Reactor.

Here’s the reactor example:

Nothing special to see here—just iterate from 0 to 9 and print a random number. The next example is more complicated. What if you want to print the numbers on a different thread? Here’s an example using JDK8 and ForkJoinPool:

Now here’s the example using reactor:

The import thing to note here is the subscribeOn operating. In this example, it’s telling reactor to subscribe on a worker provided by the parallel scheduler. The blockLast operator is called instead of subscribe because the code is running in a different thread we must wait until it’s completed or the JVM would exit.

Another thing to note is that the subscriber of the publisher can choose to run it on different threads independent of the publisher’s creator.

In this example the ‘observable’ object was returned by a method. The subscriber could either run on the current thread or do what the example did and run it using a scheduler on potentially another thread.

This next example shows how to run each random number generation in a different thread. Here’s what it looks like using JDK8’s ForkJoinPool:

This time we had to add the ‘ForkJoinTask’ objects a list and then block on them to make sure each task was executed.

Here is the equivalent example using reactor-core:

This example uses a flatMap to map and merge the results of another Publisher together. The publishers created inside the flatMap—in this case a Mono that prints a message—is told to subscribe using a scheduler. This causes each of these different Monos to run on a different thread. The main publisher is then blocked until everything is finished. What’s interesting about this is that the Monos could be anything—a synchronous Mono that prints a message, a runnable that prints a message on a different thread, or an off-box network call. They are all treated the same.

The final example does work on two different threads. The first thread generates a random number, prints it and then makes it available to another thread. The second thread then adds up the random incoming numbers printing an intermediate total, and then a final total after it gets 10 random numbers. Here’s what the example looks like in JDK8 using the ForkJoinPool:

Things are starting to get trickier here. We must use a latch and count how many receive to make sure that we have ten items. Also, we needed us some kind of thread-safe queue to send data between the threads.

Here’s what the example looks like with reactor-core:

Before we get to the publishOn operators let’s look at the map, and reduce operators. The map operator takes an integer generated by range, and then returns the random number that is generated. This is then passed to the reduce operator. In this case we are just adding two numbers, and it will add all the different random number as they are generated. At the end we call block to wait until everything is done.

This example uses the publishOn method. The first publishOn method is before the map operator. This makes the map operators happen on another thread. There is another publishOn right after the map operator and right before a reduce operator. This will cause all the reduce operations to also happen on a different thread. There isn’t another publishOn before the doOnNext, so it happens on the same thread of the reduce operator.

It is in more complex examples like this that Reactive Streams schedulers really shine. Rather than having to spend time reasoning about complex and potentially buggy multithreaded code using Java's concurrency utilities, having understood the scheduleOn and publishOn operators it is a simple matter to introduce parallelism where is it needed in a way that it both concise and easy to reason about.

In a monolithic codebase, especially a blocking one, this is easy. You can call a method or a function that returns a list, and then iterate over the items in the list. If you want to do something with the items in the list, you call another method.

This simple task is not so simple in a microservice environment. No matter how you slice it, your application must deal with the distributed nature of microservices. It doesn’t matter that your code is blocking–your code is now running on a different CPU then it was before. That function call you made may take many thousands or millions of times longer to return, and fail in new and unpredictable ways — if it returns at all.

What makes fan-out and composition such a challenge for developers to deal with? Does it need to be so hard?? I would contend that how we handle communication between services is at the root of the increased complexity.

To demonstrate this point, let’s look at communication in an office. If you work in an office with 4 people, it’s easy to communicate. There are 24 different ways for you to communicate–not so bad. But what happens when number of people in the office grows? What happens when you go from 4 to 40 people? Now, there are 8x10⁴⁷ different ways to communicate, more possible conversations than could ever take place!

Obviously, a communication strategy that works for a 4-person office would be a disaster and lead to chaos if used for a 40-person office. The same is true with microservices.

How often do you hear, “the network is unreliable” as an explanation for unreliable microservices? Is the network to blame? Or is it engineers applying the same techniques they used in monoliths to their network systems? Communication with microservices is not without its challenges. But, like communication in an office, it is a problem that is surmountable given the right approach.

Let’s consider some attributes that would make microservice communication easy, and would result in simple fan-out and composition…

1) Async Abstraction

First, microservices are asynchronous–there’s no getting around it. Even if your application is blocking the entire system, it is still asynchronous once you cross the binary boundary. If you don’t address this properly and try to block, you will have to use a circuit breaker like Hystrix to bulkhead requests. This creates an additional burden on the developer. What you really want is an abstraction that makes modeling async code easy.

2) Easy Composition

The second thing you want to do is to easily compose requests. Microservices constantly require the developer to take a result from one service and call another with it. When your code is synchronous and on-box that isn’t a big deal– but now it can come from anywhere and you don’t control when it arrives. As previously mentioned, this means dealing with asynchronous code. But how to deal with it? One way is to model it using callbacks and listeners. This is manageable when you only need to deal with one or two calls — but that doesn’t happen in the microservice world. You need to call several services at once, asynchronously, and then combine the results. So not only do you need something that makes dealing with asynchronous code easy, you need something that makes composing asynchronous calls easy as well.

3) Streaming

The third thing you need to deal with is collections of data. In a monolith, it is very easy to return a list of data, or an iterator. This gets trickier once you move off-box. There are various techniques to deal with this, but many of them are leaky abstractions, exposing details of the underlying implementation that should ideally be hidden away. Developers now have to worry about keeping track of paging or dealing with block data. What you really want is an abstraction like a stream.

4) Application-level flow control

The forth thing you’re going to need is application-level flow control, i.e. back-pressure. There are lots of examples of byte-level flow control and they work very well for the problem they are intended to solve. The problem is, they don’t necessarily protect applications.

There are many cases where the application can’t process data as fast as it arrives. If there isn’t application-level flow control, then the developer is left to deal with it themselves. This leaves you constantly having to tune queue sizes, circuit breakers, and rate limiters. What you really want is for your microservices to communicate transparently to the developer and decide how much traffic they can handle.

5) Load-balancer designed for Microservices

The fifth thing you need is a load-balancer designed for microservices. Simple round-robin or least-loaded strategies can cause thundering herd problems and cascading failures that can bring down your entire application.

Consider a service with several healthy instances and just one unhealthy instance. Under normal conditions, the median response time is 50ms with a 55ms p90. The unhealthy instance has 5ms p50 with a 550ms p90. Although the unhealthy instance has a better median response time, it will return 10 times slower one out of every 10 calls.

This represent a big problem when you think about how microservices work. In scenarios with lots of fanout and composition, the tail latency of the slowest service tends to dominate the overall response time. For example, you have two services A and B where A makes 5 calls in parallel to B for each time A is called. Service A’s processing time is only 1ms, and B’s processing time is one of the latencies mentioned above. If our load balancer is using a simple round-robin strategy, A’s latency will be completely dictated by B’s latency. If one of four instances of B is unhealthy, there is a 12% chance that A’s response time will exceed 550ms, making A’s latency profile even worse than the unhealthy service B! As we can see above this has an out-sized impact on the upstream calls in a microservice system. This can be mitigated through the use of predictive load balancing, which keeps track of a service’s performance over time and penalizes unhealthy instances as soon as they are detected.

6) Cancellations

The sixth thing you need is the ability to cancel work. Since microservices constantly fan-out calls to multiple services, it’s not uncommon that one of the services you call gets an error. Without cancellations, you will waste time and resources on the servers processing unneeded requests. This can get bad enough that it leads to outages.

Another scenario for cancellations deals with streaming. In a monolith, you can call a function or method and return a list. If the list has 100 items but you need the top 5, you can iterate over them and take the first items. The same thing can be done with streams when you have cancellations. Consume the first 5 items emitted from the stream, and then cancel the request so the other service stops emitting data.

7) Routing

The seventh thing that you need is an easy way to route between services, and methods within a service. In a monolith, this is easy to do using namespaces and libraries. If you want to call a method, you just reference a namespace, and then call a function within that namespace. You need something similar for microservices to make them easier to use.

8) Discovery with presence notification

The eighth thing you need is a way to discover when services are present and ready to send or receive requests. In the monolith world, this was done with DNS and maybe with a load-balancer. In the microservice world, where everything is async, you want to ask for a service and then be notified of its presence when it’s ready.

To recap, the 8 items are:

1) Async Abstraction

2) Streaming

3) Application-level Flow Control

4) Easy Composition

5) Load-balancer designed for Microservices

6) Cancellations

7) Routing

8) Discovery with Presence Notification

In my next blog post I will detail a service that shows how these attributes can be used make a complicated service call graph simple.

A Failure to Communicate was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.

Part II

The natural habitat of microservices is the cloud. This is more than just a trend. Because they are designed to scale independently, microservices take advantage of how cloud computing works. Cloud providers let you deploy microservice architectures quickly and easily. With the click of a button, you can spin up 1000 instances. The problem is that all thousand instances you just spun up need a way to communicate.

In Part I of this post, I examined three different ways microservices can communicate: Proteus RPC, gRPC, and HTTP (represented by Ratpack). The test was a simple ping/pong style test. But, the test was run on a laptop — hardly a realistic location for real-world deployments. This week, we move the test to where microservices live. Off to the cloud…

The Test

I was lucky enough to run this test using the greatest ever cloud provider, Azure. And by luck, I mean I had free credits. I used F16 instances — they’re tailored toward compute and have, you guessed it, 16 cores. The operating system was Ubuntu 16. More information can be found here. I kept the scenario simple again — we’ll save the fancy tests for later.

The test was still a simple ping/pong test, except this time, the client and server were on separate VM instances. Each client created at least 16 connections to the server. Netty roughly assigns incoming connections to an event loop. Since Netty creates one event loop per core, 16 connections should be enough to fully utilize a 16-core CPU.

On each run, the client sent 100,000,000 messages to the server. Two memory settings were used: 2gigs and 128megs. Running with 2gigs is typical of microservice instances, whereas running with 128megs demonstrates performance in a memory constrained environment such as a mobile phone. Both the client and server had the same memory settings. The test was run 5 times with each setting. The best run results from each run were kept. HdrHistogram was used to record the results.

Ratpack

Like last time, we’ll get a baseline using the HTTP/1.1 REST approach. HTTP was represented by the Ratpack server and Reactor-Netty client. HTTP/1.1 cannot multiplex messages. This is a fancy way of saying you can only send one message per connection at time. More than 16 connections are required for good performance. Reactor-Netty does this by default. I didn’t need to make any charges to either the client or server from the last test. With 2gigs RAM, Ratpack got 90,754 RPS and 11.8ms p995 latency. With 128megs Ratpack got 76,910 RPS and 16.3ms p995 latency. Restricting memory didn’t dramatically affect performance or latency.

gRPC

Next up was gRPC. gRPC only needed 16 connections to the server. This is because HTTP/2 can multiplex messages. I improved on the test from last time and used their StreamObserver API rather than the Futures-based API. It’s a shame Google chose not to support Reactive Streams. I still had to wrap their object to use the standard Reactive Streams interface. Additionally, gRPC doesn’t automatically detect the Linux epoll API, which generates less garbage and generally, improves performance compared to the NIO based transport — so I had to add that as well. To do this, I had to dig through gRPCs benchmark code. Why epoll detection isn’t automatic is beyond me! With 2gigs, gRPC got 699,903 RPS and 15.4ms p995 latency. This is an improvement over HTTP/1.1. When you lower the memory to 128megs, things go downhill quickly. gRPC creates lots of garbage. With 128megs, throughput is 5 times worse. Latency at p995 is 4 times worse. It only got 128k RPS with a p995 of 62.5ms.

Proteus RPC

Finally, I tested Proteus RPC. Proteus RPC runs on top of RSocket. I used TCP RSocket again. RSocket is also multiplexed, requiring only 16 connections. Proteus RPC automatically detects epoll. I updated the client to create a connection per CPU. There is a load balancer with connection pooling, but that will be used in a later test. Proteus was able to achieve over 1,000,000 RPS with 2 gigs of ram. It got 1,134,246 RPS with a 3.5ms p995 latency. Proteus is nearly 50% faster with 150% better latency than gRPC. Proteus with 128 megs performed better than gRPC did with 2 gigs of ram. It got 751,150 RPS and 11.8ms.

With 11 times less memory, Proteus RPC is 140% less latent and faster.

Again, the communication protocol that was specifically created for microservices, not using a browser protocol, performed better. It outperformed all other competitors with less ram. This brings us to an interesting point. There is a misconception with Java that it has problems with garbage collection. Java doesn’t have a GC problem — it has a problem with developers making too much garbage. The people who wrote Netty knew this. Netty provides a great byte buffer library that recycles objects to prevent waste. Proteus and RSocket go the extra mile to use this. They use zero copy techniques to prevent creating extra garbage. To drive the point home, another test was run with Proteus RPC using a 92 megabyte JVM. Think about it — a JVM test with less than 100 megabytes. I tried running the same test with gRPC, but it crashed with OutOfMemory exceptions. With 92 megabytes Proteus got over 548,000 RPS, and 15ms p995 latency. Bells and whistles can always be added, but the core of a library is hard to change. Proteus RPC has a demonstrably superior core.

Conclusion

As awesome as 1 million RPS is, that isn’t the whole story. So, what is? Well, a couple of things. First, when you build your application, the libraries you choose incur overhead. This includes your RPC. You don’t want an RPC library greedily hogging all the memory and sucking up precious CPU. The more efficient it is, the more resources you have for what matters to you. Secondly, superior performance results create a safety net for your application. Taking up all the CPU and memory leads to dangerous situations where your production systems will crash. If you’re running a resource hog, you don’t have room to deal with dangerous spikes in traffic and latency. Proteus RPC is efficient. It leaves room for what you care about. With Proteus RPC’s superior performance, latency and traffic spikes don’t affect you anymore. You have room to breathe. Everything just got a lot easier.

If you’ve been programming long enough, someone’s quoted Donald Knuth to you:

“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.”

What they never mention is the second part — the important part — what you optimize:

“Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; after that code has been identified.”

The first sentence bears repeating — “Yet we should not pass up our opportunities in that critical 3%”. What is the critical 3%? In a microservice architecture, where they live and die by communication, your RPC is in that 3%. If you care about your microservices, try Proteus RPC.

The tests from Part I and Part II produced interesting results. They were a lot like drag racing — they showed how fast each RPC went. Now we know the speed. The final results are include below. In Part III, we’ll see how they handle more realistic scenarios with multiple clients and servers. Next time we’ll include stats about CPU utilization, network utilization, and maybe a little chaos testing…

Appendix

The source code from test can be found here.

The request per second results from each test compared side-by-side. gRPC and Ratpack were unable to complete the 92 megabyte JVM test. They failed with out of memory errors.

Latency distribution comparison of Proteus RPC, gRPC, and Ratpack with a 2 gigabyte JVM. The graph is in microseconds, and captures up to the p99.99 percentile.

Latency distribution comparison of Proteus RPC, gRPC, and Ratpack with a 128 megabyte JVM. The graph is in microseconds, and captures up to the p99.99 percentile.

RPC Thunder Dome II was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.

Everyone is moving to microservices today. It makes sense. Teams no longer must fight over a giant, bloated monolith. You can deploy code as quickly or slowly as you need to, without blocking others.

In the past, when you needed to scale one part of your application, you had to scale the entire monolith. Now, you can just scale the individual services as necessary. And when a team wants to use another team’s service, they aren’t locked into a single language. They just need the service contract to call it over the network.

Most people build their microservices using REST over HTTP, and JSON. REST and JSON are convenient. They’re being used for your web site, so why not use them for services too? The problem is, HTTP was designed for web browsers. And last time I checked, servers don’t browse the web.

When you use REST and JSON, you are taking binary data and turning it into text, so it can be turned into binary again. That’s not the most efficient way for machines to talk. I know what you’re thinking — but REST is easy, what are you talking about? I assure you, it’s not.

Familiarity with something doesn’t make it easy or effective. REST is not a formal protocol, but an architectural style that was slapped on top of HTTP. PUT to insert or update? Who knows? Need a call back? Setup Kafka or Webhooks. How do you figure out what URL to call or what JSON to send? Check a swagger doc if you’re lucky, but most likely you’re talking to someone else. The semantics of browsing the web are not a good fit for application development. There is a better way.

Early microservice proponents like Facebook, Google and Twitter found a formula that works — a language-independent interface description language (IDL), RPC framework, and a code generator. Over the years, Google has released pieces of its framework.

First it released Protocol Buffers, its IDL, and serialization format. More recently, they released gRPC. Google describes gRPC as, “A high performance, open-source universal RPC framework.” gRPC follows the formula method above: IDL, RPC framework, and code generator. All good, right?

Not quite. Decisions were made that are strange for a “high performance…universal framework”. As a result, gRPC fails to move the needle on microservices communication in important ways.

Microservice architectures are asynchronous by nature. gRPC makes this difficult with a bad API. Rather than leverage a standard for asynchronous stream processing such as Reactive Streams, gRPC opts for a callback-based API. Non-trivial applications require gRPC to be adapted to a framework or users are quickly forced to deal with callback hell.

Another odd choice is to wed gRPC to HTTP/2. HTTP/2 has great new features to address shortcomings in HTTP/1.1. That’s what it’s for — to be better at schlepping webpages, not the foundation of an RPC framework. From the HTTP/2 documentation:

“This effort was chartered to work on a revision of the wire protocol — i.e., how HTTP headers, methods, etc. are put “onto the wire”, not change HTTP’s semantics”

The result is gRPC is limited by HTTP. Ironically, gRPC doesn’t even work in browsers. It’s ridiculous when you think about it. They literally created an RPC for servers using a protocol for browsers — that can’t be used by browsers!

This brings us to the final point, the “wire protocol”. gRPC’s wire protocol isn’t really a protocol. It’s a set of HTTP headers. HTTP/2 doesn’t support binary values in the headers, so you have to Base64-encode any octet data or just send strings. Instead of sending things that can be represented as binary — version number for instance — a string that requires parsing is sent instead.

Enter Proteus RPC. Proteus RPC is also designed to be a fast and universal RPC framework. It is built on top of RSocket. Unlike HTTP, RSocket was built to fully model application interactions over a network — it’s built for microservice architectures. More information about RSocket can be found on rsocket.io. Proteus is designed to take advantage of RSocket’s powerful features, but at the same time, present the developer an easy to use API.

I wanted to see how Proteus RPC performed against gRPC on the JVM. In order get to a valid comparison, I needed a reference point using REST/JSON. I decided to use Ratpack. Ratpack is a high-performance HTTP framework built on Netty. This is good, because gRPC and Proteus can both use Netty for their network transport. With all three using Netty, we can see how protocols, framework design choices and serialization, affect performance independent of networking choices.

The test described in this post is a simple ping/pong style test, run on my MacBook Pro i7 laptop. I plan to write about more comprehensive tests in later blog posts. First, 1,000,000 messages are sent to warm the JVM. Then, 1,000,000 messages are sent whose throughput and latency are measured. HdrHistrogram captures the latency. gRPC has a blocking client, but to make the tests fair, I used the non-blocking client. Both Proteus and gPRC can support Protocol Buffers so I used the same IDL for both. For the REST/JSON test, I created a simple Java object that mimics the Protocol Buffers object.

I tested REST/JSON first with Ratpack. Ratpack does not have an external HTTP client, so I used reactor-netty’s HTTP client. Ratpack’s promise API works well, so I used that on the server side. It got decent performance results — 28k RPS and a p995 of 11.9ms.

Next, I tested gRPC. gRPC’s Java non-blocking API is non-existent. The first test crashed my JVM. It ran out of memory from creating too many threads. After some quick research, I found you need to call directExecutor on your builder to prevent gRPC from using an internal cached thread pool. Even with this added, the tests didn’t run well and caused exceptions. This is because gRPC doesn’t support application backpressure, so there is no signal to prevent a server from being overloaded.

I needed some lipstick for this pig, so I used reactor-core to emulate backpressure. That worked better. gRPC got better throughput than Ratpack — 67k RPS, and better p995 latency 1.9ms. After p995, its latency hockey-sticks to the 10ms range.

Finally, I tested Proteus RPC. Proteus RPC uses a Reactive Streams-based API, making async programing very simple. Non-blocking requires zero special configuration, and it supports application backpressure out of the box. Proteus’ throughput was 91k RPS, with a p995 of 0.64ms. It was 225% faster than Ratpack, and almost 40% faster than gRPC while achieving 70% lower latency. gRPC’s p50 of 0.856ms was worse than Proteus’ p995 latency.

Using the same networking library, Proteus RPC achieves higher throughput and lower latency, without special configuration. If you are moving from REST/JSON, try Proteus RPC. I have included some graphs below, and a link to the source for my test: https://github.com/netifi/rpc-thunderdome. In my next blog post, I will repeat the same tests, but in a cloud infrastructure to see how this plays out with real hardware.

RPC Thunder Dome was originally published in Netifi on Medium, where people are continuing the conversation by highlighting and responding to this story.