Scalable Ruby — Concurrency and Parallelism Explained
And why the differences matter
Years ago, Ruby ruled the web. It was very easy to build new apps, there were many gems that solved common problems, and, therefore, it was easy to build new features.
But then things changed. Twitter struggled to make Ruby scale and switched to different platforms to satisfy the performance requirements they had. More and more voices pointed out that Ruby was too slow and unable to handle the demands of the web.
This gave rise to new technologies that scaled easier, that scaled better. Node especially satisfied this demand and created a new hype around it. Other technologies that popped up as well were Golang, Elixir, and Scala/Akka, to name a few.
Note: When I am talking about concurrency in Ruby, I will not talk about multi-process architecture or how to use load balancers. The goal of this piece is to show how to make use of concurrency in Ruby within one process, and to show the difference between parallelism and concurrency.
Does this mean that Node runs all this requests in parallel too? Actually, no. The event loop and non-blocking IO operations make it possible for the V8 to run the requests concurrently, but not in parallel.
“Wait, what?” I hear you say. “What is the difference and why should I care?”
What Is Concurrency and Parallelism?
A very accurate definition of concurrency I found is this:
“The coordination and management of independent lines of execution. These executions can be truly parallel or simply be managed by interleaving. They can communicate via shared memory or message passing.”
When talking about concurrency I will talk only about the managed by interleaving type of concurrency.
Suppose we have two requests from clients. One of them wants to update some data on an employee, and the other one wants to access a list of all departments in a company. How does the V8 engine handle such a situation?
The first request will be handled first. For example, the server checks if the user is allowed to save the customer. To do this, the app makes a request to the database. And now, it happens! This request to the database is non-blocking, which means that the app doesn’t wait for the response and continues. Since the current request has nothing to do until the database is responding, it notifies the Node scheduler that it is idle. Now Node can handle the second request.
By the way, this is the reason why a JS developer has to use async/await, promises or callbacks. All of these techniques allow the JS engine to continue in it’s flow and handle the response after the database sends the data, while the developer has the code in one place.
What does this mean for Ruby?
Since (nearly) all web apps have a lot of network requests to do, the main thing that throttles the performance are the request and the waiting for the response in Ruby, since it uses a blocking IO as default.
Later we will see what we Ruby developers can do to make concurrency work.
Here is the definition:
Truly simultaneous execution or evaluation of things
Parallelism is easier to understand because it works just as you would imagine. Two requests come to the server, two requests are handled in parallel, at the same time, from the server. For computers, this means that two cores are working simultaneously.
How To Achieve (Non-Parallel) Concurrency in Ruby
And now to the goal of this piece: How can we achieve concurrency in Ruby?
As mentioned above, it is possible, but it is not so easy right now. Ruby doesn’t have async/await, it doesn’t have promises, and to use non-blocking IO you would have to write your own ORM Adapter.
“But we have threads!” I hear you scream. Yes you are right. With threads we can use concurrency. How would you do that? Here is an easy example with Rails’ ActiveRecord:
Disadvantages compared to Node concurrency
Every thread maps to one OS thread. Therefore, it is not possible to create hundreds, or even thousands, of threads. The developer has to do more ceremony. In other words, he has to do a lot of the stuff more explicitly.
Since we are using threads and Ruby threads translate to OS threads…
Can We Make Use of Parallelism With Ruby, Too?
Unfortunately, not within one process. Ruby has something called the GVL (short for Global Virtual Machine lock — sometimes called GIL — Global Interpreter Lock). With this mechanism, Ruby makes sure that only one OS thread is currently running. This is, essentially, a global flag to show if it is OK to run code in current thread.
For IO operations, like the database request in the example above, this is perfectly fine, since the thread is waiting for the database and your OS is putting it to sleep anyways. Therefore the second thread can be processed — at least until it is put to sleep as well because of a database request, or because the OS switches threads.
But for Ruby code it is a little bit different. You can use threads with Ruby without IO bound operations, but then, you will not see any increased performance. MRI Ruby would check the GVL, and only when this flag is true would it run your code.
Why would you create something like the GVL?
The reason that Ruby uses the GVL is based in its philosophy: Make programmers happy!
It is very hard to write multithreaded programs. The GVL makes this easier as you cannot (so easily) have a deadlock, or other nasty things that would occur very easily in multithreaded programs.
As a side note, Python is using a GIL too. So, this was a common thing to do for language designers.
What About Different Ruby Implementations?
JRuby can use real parallelism. There is no GVL whatsoever that would stop your code. The downside is, of course, that now you are responsible to make sure your code is thread safe, and that you avoid deadlocks and other evil stuff.
TruffleRuby also doesn’t use a GVL. There were even some discussions to go a step further by trying to put everything under a Mutex if — and only if — your code uses threads. This way you would have the advantage of a GVL, plus the performance boost from multithreaded apps. But, right now, this is still an experimental feature. Only time will tell if it will become something usable.
Back to the future — Ruby 3
Matz has recognized the problems around parallelism and concurrency. For this reason, he has announced the “Year of Concurrency” (This video is available only in Japanese). How can Ruby achieve a similar, or maybe even better, scalability to something like Node? Here are three projects that want to make this possible
Scalability — Does It Really Matter?
Most of the time the answer is no. When you start a new project or app, your app simply has very, very few users. Ruby (also: Rails) is perfectly fine to handle average amount of users without problems. Many companies have shown this. Ruby can even handle many users, as Basecamp, Shopify, GitHub, and AirBnB have shown. And, if you ever come to a place like Twitter, where you need even more performance, you have way more resources by then. More money to break up a monolith, more domain knowledge, and also more technical knowledge because of the employees you would probably have by then.
The future of (MRI) Ruby seems to push this barrier even further away, so you can happily code with the language you love and are so productive with!