Java Concurrency — Threads (Part 1)
Threads
Background
One of my goals of 2023 is to write. Finally, I am writing my first post. So, where to start with?? I decided to share my learnings on Java Concurrency, this will span across multiple posts.
I assume, readers have basic understanding about process & threads and only Linux OS is considered.
Java Thread == Native OS thread
Thread is a lightweight process which has it own stack, registrar, program counter i.e, thread has its own execution path with in a process
Thread th = new Thread(<Some Runnable Object> , <threadName>);
JVM uses the underlying native OS threads to support multithreading in java, i.e. when a new thread is created in java as above, JVM internally creates a native OS thread. So, the threads are managed by the underlying OS & JVM just maps the java threads to OS threads. Thread scheduling, time slicing etc. are handled by underlying OS.
Let's use the below code to investigate on the native OS threads. On executing the below program the main thread just spawns the 3 threads namely printThreadId-1, printThreadId-2, printThreadId-3and goes to sleep infinitely, and each of the thread just prints its name, threadId and goes to infinite sleep (After playing around, don’t forget to kill this infinitely sleeping process)
Now, we can see the underlying OS threads(i m using ec2, which uses Linux distribution OS). In the below image, after running the java program, it prints the process ID, which can be obtained using ps as well.
Using the ps cmd with option -T ps -T -p <pid> you can list all the threads of the process
we can see that the OS threadId printed (circled in green). Remember, JVM has its own threadId, which is printed as part of code. JVM just maps its own threadId to the OS threadId
So, here for thread with name printThreadId-1
JVM threadId 12 == Linux threadId 16540
In the above example, it's one to one mapping, but it's not the case always, depending on the JVM implementation it differs
Thread Parallelism & Context Switching
Do threads really run in parallel? This is the most common misconception about threads. Threads do not run in parallel. It's the OS (or scheduler), which creates the illusion that the threads are running in parallel.
Let's take a single core machine, When we have multiple threads ready to be run, how one core is allocated? Who decides which thread is to be run ? It's the Scheduler which is part of kernel which decides which thread is to be run now & how long it could be run(time-slice).
When a thread is BLOCKING (to obtain monitor of shared resource)/WAITING (waiting to get a response from a IO) instead of wasting the CPU cycles by waiting, the scheduler schedules the other threads which are ready to run. This creates the illusion that, threads are running in parallel.
Scheduler allocates time for the running threads to ensure fairness in scheduling. Once the time is over, current thread is put in the queue and other threads in RUNNABLE state are given chance to run
The process of storing the context of the currently blocking or waiting thread and scheduling the other thread is called as context switching. Context has the information about the line currently being executed, thread local variable values etc.
You can see the context switching happening in a machine using below command vmstat
cs gives the number of context switches occurred.
If you want thread specific information, it can found in thread status file. cat /proc/<processId>/task/<threadId>/status, at the end of the file you can see
voluntary_ctxt_switches : when a thread is blocked or waiting, the scheduler will schedule other runnable thread
nonvoluntary_ctxt_switches: even if the thread is executing, the time allocated for the thread is over. So scheduler will preempt this thread and schedule other thread
Hence, at any given point, only one thread will be using the CPU resources in a single core machine. If your system have multiple cores,
number of threads executing at any given point == number of cores in the machine.
We are basically limited by the hardware here. Check the number of cores in your machine using nproc command.
How many threads are required?
Our basic instinct is to increase the number of threads to speed up the execution, What happens if I keep increasing the no of threads to speed up my execution ?? How to know the limit?
Having too many threads might lead to frequent context switching, and the CPU would spend most of its time in saving and loading the context rather than actually executing the threads.
As per Concurrency in Java book,
number of threads required = number of cores * target CPU utilization * (1+ wait time / compute time)
- Target CPU Utilization — how much of the CPU you wanted to be used
- wait time — Time a thread spends in waiting for external resource to be available
- compute time — Time a thread spend in doing actual computation
Example : Let's say, we have a service running in 4 core machine, which calls an external API and process the response, and You want your max CPU utilization to be 40%
- latency of the external service API(P95) = 200ms (wait time)
- time taken to process the result = 20 ms (compute time)
number of threads = 4 * 0.4 * (1+ (200/20)) = 18 (rounded)
You can get the number of cores the machine by at Runtime using Runtime.getRuntime().availableProcessors() and it's always recommended to have dynamic thread count (thread-pool size) based on the number of cores.
This gives you a rough idea of how many threads to use, but in reality it's very difficult to measure the number of threads required by your system, so it's recommended to start with this count and tune your system according to your load testing results.
Note : If you have very compute intensive tasks (which is not the case mostly), it is advised to have no of threads threadcount(usually thread-pool size) == no. of cores + 1, because even with compute heavy tasks, there can be page faults which require a IO to load page to memory, during that IO wait, other thread can use CPU cycle for execution
