Determine the ideal thread pool size-Java Concurrency

Many times java programmers stuck in situation while doing java programming that what should be the ideal thread pool size ?

The answer to this question is not straight. Because there are few parameters on which it depends to determine the size of thread pool.

Thread Pool is known as a pool of worker threads. Worker thread is a thread which accepts a task, complete it and come back again to the thread pool to accept another task.

The size of the thread pool primarily depends on the below two factors:

Number of CPU Cores

A single core CPU will run one thread at a time.A normal desktop generally is a quad core that means there are four cores in a CPU. While a cloud or server may have as many as cores in CPU.

If we consider the hyper threading then a single core CPU can have multiple processors.

You can find the number of processors by using below java code

int poolSize = Runtime.getRuntime().availableProcessors();

It is also possible that a server has many cores but the your software application has only access to few cores. So in that case, the available cores to the application will decide the number of thread pool size.

Type of tasks

There are two type of tasks:

• CPU Bound- Tasks which involves mathematical calculations.
• IO Bound- Tasks which involve communication to other application through network calls like database, web services

CPU Bound

Let's consider there is one CPU core and one thread is running which has 2 tasks submitted. Then one task is submitted to thread one and once that is completed then other task is submitted. Between the submission of two tasks there should not be any time gap to achieve the maximum utilization of CPU.

IO Bound Tasks

Tasks which do communication to other applications through network calls like web services, database, outside caches, micro services etc.

In the above diagram, there is 1 thread and 1 task is submitted to it. When this task wait for IO operation to complete then CPU becomes free and idle. When IO call gives the response back then it again starts working until task is not completed.

During the idle time we can initiate 1 more thread and make it running so the maximum utilization of the CPU is achieved and thread 1 could be in wait state until the output is received back from IO call.

Therefore, in case of IO bound tasks with one core CPU, you can increase the number of threads and could gain the maximum utilization of CPU.

Formula to calculate the number of thread count for optimum utilization of CPU.

Ideal thread Count= Number of Cores * [ 1+ (wait time/CPU time)]