So you think you know Java Multi threading!

The Himalayas — from an iPhone

Having worked as a Java programmer for more than half of my career, I am often surprised (and sometimes not) as to how many developers don’t really understand how multi threading in the JVM works. This could be in part because we have become used to writing stateless services or the multithreaded aspects of the code are hidden behind the framework ( @async annotations in Spring and the Executor Framework in Java). This post is quick refresher (not to be substituted for an authoritative source) on certain aspects of multi threading , specifically synchronisation and thread safety and how the JVM handles it. It talks about the familiar Producer — Consumer problem and how to identify “shared state” with examples.

The Producer Consumer Paradigm

Essentially a Producer Consumer paradigm is one in which a producer (in this context a thread) produces an output and the consumer consumes the output. The output may be stored in a queue or a shared variable. With Java 5 and above, the java concurrency package has provided a bunch of classes to help abstract out the underlying semantics of safe access by providing the Executor Service Framework . This framework provides classes that allow you to work with task Executors that take an Runnable and execute it — hiding the synchronisation constructs from the programmer. The executor will return a Future that can then be used to retrieve the result of the task.

Think about the following scenario — I have a producer which needs to produce a series of numbers (e.g 1..100) and a consumer which needs to read the number as and when it is produced. If there are multiple producers and multiple consumers and the rate of consumption and production can differ, then it makes sense to put the produced output in a queue. But if our requirement is just reading the shared state of a ‘Counter’ variable, we do not need to put this in a queue. Identifying what is shared is the first problem in figuring out what needs to be protected for shared access.

Operations to Java variables are not atomic. An operation such as addition can be susceptible to race conditions if multiple threads read the value of memory location before the write operation is finished. The Java Memory Model (JMM) defines the specifications for this. Volatile declarations, AtomicIntegers, AtomicLongs and AtomicReferences help ensure that the write is atomic but that does not necessarily mean the thread is reading the right value. It might still be reading a stale value if synchronised access is not taken into account. Adding a volatile keyword in front of the variable tells the JMM that the write to the variable has to be atomic. AtomicIntegers, AtomicLongs and AtomicReferences also achieve the same result of atomicity but have additional helper methods to help the notion of whether the write is required and to read the value after the write.

In Producer Consumer case defined above, the producer increments the value of a shared counter , and the consumer reads the last incremented value of the shared counter. So we need to ensure that the shared counter at any point is only accessed by one thread to read its current value or write its next value.

An analogy would be to imagine a warehouse in which the only way to get to the contents is through a door which is locked. To put an item into the warehouse, one has to obtain a key and open the door and put the item into the warehouse and once the item is in the warehouse, the key is then returned. To get an item from the warehouse, the key has to be obtained , the door unlocked and the item retrieved. In this above case the warehouse or the CounterObj is shared between a producing thread (Thread A) and a consuming thread (Thread B). The lock is mutually exclusive (a mutex) that is, at any point in time either the producer or the consumer can obtain the key to the lock. They cannot both acquire the same key at the same time. Once the job is done, the key (or the access to the lock ) has to be released by the owning thread.

Now what does the thread do when it puts the item in the warehouse and now has to release the key/lock? Does it sleep or wait? If the thread sleeps in the synchronized block or as per our analogy the warehouse — it has not actually released the key to the warehouse. Its still holding on to the key and and is not doing any work except release the use of the CPU. This means no other thread can now acquire the key. So effectively no other thread can now get into the warehouse and and add or get an item if the lock is an exclusive lock (mutex) unless the first thread gets interrupted from its sleep. Coming back to thread constructs — the right option would be to call the lock object’s wait method which releases the lock so that another thread can enter any synchronised block protected by the same lock. In the warehouse analogy — this is like exiting the warehouse and giving the keys back and queueing at the entry of the ware house to acquire the key once it becomes available again.

Note that many areas or synchronised blocks can be protected by the same lock object. If the state that each of these blocks is protecting is not the same, consider using a new “lock” object for synchronising the access. Every object has an intrinsic locking mechanism that is part of the Java Object Inheritance (wait and notify) and is internally associated with a lock access count and a wait set. When a thread acquires the object’s lock , the lock access count gets incremented. In the case of mutually exclusive locks, the lock access count can only be 1 or 0. If the lock access count is 1, then another thread which needs to enter a synchronised block held by the object’s lock, will get added to the object’s wait set. A thread gets added to an object’s wait set either at the entry of the synchronised block protected by that object, or by calling the wait method on the object. Once the shared access block is executed, the thread holding the lock will call the object’s notifyAll or notify to let the threads in the wait-set know that the lock can now be acquired again.

Here is a Code snippet with the Producer Consumer entities mentioned above.

If the CounterObj is declared as volatile, then the write update is ensured to be visible to threads reading it. If we changed the program a little, to remove the wait notify block and instead use Thread.sleep(1000) without the synchronised block, we would still see a correct output but intermediate updates would be lost to the Consumer thread.

If your application needs state but is OK with multiple threads seeing different versions of the state, then ThreadLocal confines the value to a single thread. What is the difference between using ThreadLocal and volatile you might ask. Using volatile provides a weak level of synchronisation and reads and writes are atomic. volatile ensures that all thread read the most current value from main memory (not from the cpu cache or registers) and flushes back to main memory — however compound operations such as read and then increment i++, i==, i+=x etc are not guaranteed to be atomic. Using ThreadLocal however, each thread has its own value of that variable which does not need to be synchronised.

But what is shared state ?

If an object (a Servlet or a Spring Bean) can be accessed by multiple threads and has class level variables that are mutable, then that object has shared state. Spring Containers, Web server implementations are inherently multi threaded. The beans or servlet instances handle multiple requests at the same time , each request originating from a separate thread. If an object is created by an entity and then accessed in two separate instances of different types through a reference parameter, it is still shared state. If objects are created within the method being executed, those are not shared as each Thread has its own stack frame and local variables are placed on the stack frame. Consider the code snippet below to illustrate the above.

In Conclusion

Having been bitten by multi threading bugs in the past, I have been a careful Java programmer and looking out for shared state is part of my review checklist. Doug Lea’s book on Concurrent Programming in Java is a must read for every Java developer to read and read again. If you haven’t ever encountered a multi threading error that caused an unpredictable behaviour of your application, potential thread safety issues can be bit hard to visualise. Hopefully this post would have helped in that visualisation!

Thanks to inputs from Abhijit Belapurkar !