Ballerina Concurrency: No More Ticking Time Bombs
This article was written using Ballerina Swan Lake Update 10 (2201.10.1) but is expected to remain compatible with newer versions.
Ticking time bombs
Writing concurrent programs is hard — very hard. Don’t believe it? Try determining whether the programs below are safe to execute concurrently. They are written in Java but apply to many general-purpose languages.
public class X {
private int lastIdUsed;
public int getNextId() {
return ++lastIdUsed;
}
}public class MyClass {
double foo = 0.0;
public double getFoo() {
return foo;
}
public void setFoo(double foo) {
this.foo = foo;
}
}First example is extracted from Clean Code and here is what it has to say about that piece of code.
A quick answer, working with just the generated byte-code, is that there are 12,870 different possible execution paths for those two threads executing within the getNextId method. If the type of lastIdUsed is changed from int to long, the number of possible paths increases to 2,704,156.
Well, writing programs ensuring concurrency safety is a next level problem. Like it or not mistakes will slip in to your program. Eventually turning them to ticking time bombs waiting to explode in your production deployment.
I once received a call from my lead around 9 PM, informing me of an urgent problem. For some reason, user details were getting mixed up in one of the production services. I was given the logs, the code, the issue to address, and the deadline: ASAP — no pressure. So, I delved into thousands of lines of code written over the years, searching for the shared mutable variable hidden somewhere. I managed to find it and resolve the issue by 4 AM. It was a race condition — a type of bug that’s hard to track down and often appears in hard-to-reproduce runtime situations. Trust me, you don’t want them.
I am sure if you have been working with services/APIs you must have had similar experiences.
Ballerina being a programming language born in the world of services wants to solve this problem. Therefore, it takes a slightly different approach to ensure concurrency safety. It sort of ensures that you never deploy time bombs to production environment by detecting race conditions during development time than runtime. In other words, you can write robust services using Ballerina and that means no more ticking time bombs.
Ballerina’s approach to concurrency
Before you learn about any programming language concurrency, you need to know that the heart of concurrency safety is about managing access to shared mutable state.
Whenever more than one thread accesses a given state variable, and one of them might write to it, they all must coordinate their access to it.
Let’s see what tools Ballerina has to facilitate the above while ensuring race conditions are detected during compile time not runtime. Let me repeat it as it is very important, during compile time not runtime!
Consider below service written in Ballerina.
import ballerina/http;
import ballerina/uuid;
map<json> userStore = {};
service / on new http:Listener(9090) {
resource function get users() returns map<json> {
return userStore;
}
resource function post user(@http:Payload json user) returns map<json> {
userStore[uuid:createRandomUuid()] = user;
return userStore;
}
}Okay, the first question is: is it concurrency safe? as I explained before, this program access shared mutable state and one of them is writing to it, therefore, it is not concurrency safe. In fact Ballerina compiler detects it let you know. Furthermore, it won’t let you execute it concurrently but instead give you the warning below.
Okay, now we know there is a problem. We always need our services to run concurrently. We want each request to be isolated from on another as if they live in their own world.
If you know even a bit about concurrency, you know that the first thing we need to do is guard the critical section. How do we usually do that? By using locks! Like many other programming languages, Ballerina has locks too. So, let’s use them.
import ballerina/http;
import ballerina/uuid;
map<json> userStore = {};
service / on new http:Listener(9090) {
resource function get users() returns map<json> {
lock {
return userStore;
}
}
resource function post user(@http:Payload json user) returns map<json> {
lock {
userStore[uuid:createRandomUuid()] = user;
return userStore;
}
}
}Although the critical section is guarded with a lock, we still see the warning below. This is because the shared mutable state is not isolated.
So, what is missing? isolated . Let’s add it in front of userStore variable which is the shared state.
import ballerina/http;
import ballerina/uuid;
isolated map<json> userStore = {};
service / on new http:Listener(9090) {
resource function get users() returns map<json> {
lock {
return userStore;
}
}
resource function post user(@http:Payload json user) returns map<json> {
lock {
userStore[uuid:createRandomUuid()] = user;
return userStore;
}
}
}When you add that keyword, you get more errors. To resolve those errors, you need to understand what is the discipline isolated keyword wants you to follow.
What is isolated ?
The isolated keyword essentially makes the userStore variable an isolated root. In simple English, it works like this: imagine the userStore as an island that isolates all the users, with only one dock providing access to them. Here's an example to illustrate this:
This means if you guard the dock with a lock, you guard access to the entire users of that island. A simple restriction but with a massive effect on writing comprehensible code. On the other side, this radically simplifies the compilers complexity to detect race conditions during compile time. Therefore, the discipline of one dock for one island is enforced by Ballerina and in return it gives you compile time validation for race conditions.
The summary of Ballerina’s approach to concurrency is as below,
Whenever you don’t follow this discipline, complier disciplines you by returning errors. It is a discipline that adds restrictions on accessing shared mutable state.
Ballerina way of writing concurrent programs
Now, with that in mind, how do we resolve the previous errors? We can create a fresh copy of the user and put it into the map. Since we’ve created a new copy, it guarantees that there are no other references to it except from userStore. We need to do the same when returning userStore data.
import ballerina/http;
import ballerina/uuid;
isolated map<json> userStore = {};
service / on new http:Listener(9090) {
resource function get users() returns map<json> {
lock {
return userStore.clone();
}
}
resource function post user(@http:Payload json user) returns map<json> {
lock {
userStore[uuid:createRandomUuid()] = user.clone();
return userStore.clone();
}
}
}Nooww, we can run it concurrently and also compiler can validate race conditions during compile time.
Imagine multiple docks for one island
If we were allowed to have multiple docks for one island, you should be able to do the below.
import ballerina/http;
import ballerina/uuid;
isolated map<json> userStore = {};
json latestUser = {};
service / on new http:Listener(9090) {
resource function get users() returns map<json> {
lock {
return userStore.clone();
}
}
resource function post user(@http:Payload json user) returns map<json> {
lock {
userStore[uuid:createRandomUuid()] = user.clone();
latestUser = userStore[uuid:createRandomUuid()];
return userStore.clone();
}
}
resource function get latestUser() returns json {
return latestUser;
}
}It would look like the below. Now you have multiple docks to reach User5 .
This type of programs can become exponentially complex very fast. Complex enough that even compilers can’t detect race conditions during compile time. Therefore, it is restricted in Ballerina.
Okay, with that in mind: how can you make the above program run concurrently? As earlier, you first need to isolated the latestUser and then do required changes. Doing so would result in something like the below.
import ballerina/http;
import ballerina/uuid;
isolated map<json> userStore = {};
isolated json latestUser = {};
service / on new http:Listener(9090) {
resource function get users() returns map<json> {
lock {
return userStore.clone();
}
}
resource function post user(@http:Payload json user) returns map<json> {
lock {
latestUser = user.clone();
}
lock {
userStore[uuid:createRandomUuid()] = user.clone();
return userStore.clone();
}
}
resource function get latestUser() returns json {
lock {
return latestUser.clone();
}
}
}Noticed something strange in the second resource function? There are two lock statements instead of one. This is because it’s not allowed to use more than one isolated variable inside a given lock statement. If you could, it would undermine the isolated root concept, which is the cornerstone of detecting race conditions at compile time. How? By taking a user from userStore and assigning it to latestUser. Remember, we must only have one dock for one island.
Now that you have some sense about Ballerina concurrency and the concept of isolation for shared mutable state. Let’s see some of the best practices and on Ballerina concurrency.
Best practices
Think Services, Think Concurrency
They are like ink in water: inseparable. Therefore, you cannot think of one without thinking about the other. It is simply not practical. This means you must think of concurrency safety from the start of the service development. It shouldn’t and can’t be an afterthought.
Therefore, if you are writing a service start with marking the service and it is resource functions as isolated . Yeah, you can do that and when you do so, you will immediately get compilation errors instead of warnings, whenever there is a race condition detected. This will force you to think of concurrency from the start. Otherwise, you will continue to write the buggy code and by the time you realise it, you may have to do a lot of changes to your code. Btw, isolated functions are infectious, meaning, if you make one isolated you must make all the nested ones are also isolated.
Note: If you don’t mark the service as
isolatedcomplier will try to infer if the service is in fact isolated. However, complier won’t force you to make your service isolated.
Use shared immutable state by default
Remember concurrency is all about managing access to shared mutable state, therefore, try to avoid mutable state in the first place. You can use readonly to do that. Also, as a habit, make variables final.
Use copies of data
Try to use a copy of the data instead of mutating the shared data. Of course, this have to be done considering size of the data being copied. But most of the time allocation is much cheaper than a lock. So, don’t be afraid to copy things!
Isolate into one
If you have two or more isolated variables that are coupled to one another try to merge them into one. A good way to identify this is to look for logic that often require multiple lock statements instead of one. That could be an indication that isolated variables accessed in the locks are coupled to each other.
This could have performance impact but your program would work. In case it drops performance below the requirement, you can think of ways to segregate the lock. Remember, first make it work, then make it fast.
Stick to Single Responsibility Principle (SRP)
Though this is not specific to Ballerina you can try following SRP and encapsulate shared mutable state related code into one location using modules or objects. That part is covered in the next section.
So, based on the best practices mentioned above, let’s try rewriting the code.
import ballerina/http;
import ballerina/uuid;
isolated map<json & readonly> userStore = {};
const string LATEST_USER = "latestUser";
isolated service / on new http:Listener(9090) {
isolated resource function get users() returns map<json> {
lock {
return userStore.clone();
}
}
isolated resource function post user(@http:Payload json & readonly user) returns map<json> {
lock {
userStore[uuid:createRandomUuid()] = user;
userStore[LATEST_USER] = user;
return userStore.clone();
}
}
isolated resource function get latestUser() returns json {
lock {
return userStore[LATEST_USER];
}
}
}I think now you are ready to get started with writing concurrent Ballerina services. By now you should have some sense about the concept of isolation in Ballerina and how it helps Ballerina to detect race conditions during compile time not runtime.
You either have the pain of discipline or the pain of regret. Choose your pain!
Okay, it does sound like a lot of work, an additional burden, and something that slows you down. In my opinion, it is no different from having to write test cases before deploying code to production. In both cases, it slows you down in the present but allows you to move faster in the future.
Remember! no more ticking time bombs!
More on Ballerina concurrency
Isolated objects
First of all, there is no difference between modules and objects in terms of concurrency. Both can have shared mutable state and functions that access them. It is just that objects have object level boundary whereas module has module level boundary. Therefore, the same rules we discussed earlier applies for objects as well.
Here is an example of an isolated object.
isolated class DataStore {
private map<json & readonly> data = {};
private string LATEST_USER = "latestUser";
isolated function add(json & readonly user) returns map<json> {
lock {
data[LATEST_USER] = user;
data[uuid:createRandomUuid()] = user;
return data.clone();
}
}
isolated function get() returns map<json> {
lock {
return data.clone();
}
}
isolated function getLatestUser() returns json {
lock {
return data[LATEST_USER];
}
}
}Now let’s try to use it in our service. As usual, we can make the variable isolated and use it as follows.
import ballerina/http;
import ballerina/uuid;
isolated DataStore dataStore = new();
isolated service / on new http:Listener(9090) {
isolated resource function get users() returns map<json> {
lock {
return dataStore.get();
}
}
isolated resource function post user(@http:Payload json & readonly user) returns map<json> {
lock {
return dataStore.add(user);
}
}
isolated resource function get latestUser() returns json {
lock {
return dataStore.getLatestUser();
}
}
}However, if you really think about it, isolating userStore doesn’t make sense, as it is already an isolated object. Instead, you can replace isolated with final and use it in an isolated function. This also eliminates the need for the additional lock.
import ballerina/http;
import ballerina/uuid;
final DataStore dataStore = new();
isolated service / on new http:Listener(9090) {
isolated resource function get users() returns map<json> {
return dataStore.get();
}
isolated resource function post user(@http:Payload json & readonly user) returns map<json> {
return dataStore.add(user);
}
isolated resource function get latestUser() returns json {
return dataStore.getLatestUser();
}
}You might wonder, “Why do I need to make it final?" If you don’t make it final, you could update the userStore reference while it’s being used by another thread, leading to undesired results or potential time bombs. Therefore, you are restricted from doing that.
It is bullet resistant but not bullet proof
While Ballerina makes its best effort to guard against slipping code with race conditions to production, it is not 100% bullet proof. There still could be situations where getting the right answer depends on lucky timing. Following is an example for that.
import ballerina/http;
import ballerina/uuid;
isolated map<json> userStore = {};
isolated string theLastUserId = "";
service / on new http:Listener(9090) {
resource function get users() returns map<json> {
lock {
return userStore.clone();
}
}
resource function post user(@http:Payload json user) returns map<json> {
string userId = uuid:createRandomUuid();
lock {
userId = uuid:createRandomUuid();
theLastUserId = userId;
}
lock {
userStore[userId] = user.clone();
return userStore.clone();
}
}
resource function get latestUser() returns json {
string userId = "";
lock {
userId = theLastUserId;
}
lock {
return userStore[userId].clone();
}
}
}Final thoughts
If you really think about it, the evolution of programming languages is about restricting things. Structured programming restricted goto statements. OOP restricted pointers to functions. Functional programming restricted assignment. Ballerina restricts access to shared mutable state in the form of isolation. All these restrictions discipline us to write more robust code.
