Everything you need to know about Thread Pools in Java

Threads

In essence, a thread is a sequence of instructions that can run independently. They are vital for achieving concurrency in modern applications and are one of the easiest and cheapest ways to leverage the multiprocessing power of the CPU.

Thread Swim Lanes

Threads can be thought of as swim lanes, where only one task occupies a lane at any given time. Each task operates at its own speed, and the thread is dedicated to that task while it’s running.

The greater the number of threads we generate, the more tasks can run concurrently. Nevertheless, threads come with a cost. Each thread consumes at least 1MB of RAM and requires a few milliseconds to initiate. While this might not seem substantial, in terms of low-level resource management, it represents a noteworthy amount.

The primary challenge arises from CPU context switching. When a process intends to transfer execution power to a thread, it typically needs to revoke it from another thread, a operation known as context switching. In this process, the core unloads the stack frame of the current running thread, stores it in memory, and then loads the stack frame of the new thread, hence the term “context switch.” To manage this efficiently, it’s crucial to maintain the number of threads at a reasonable level, typically within a few hundred.

Spawning a thread is quite easy

Thread thread = new Thread(
    () -> System.out.println("Executing some task...")
);

thread.start();
thread.join();

First, we create the thread and assign it a Runnable. This initializes the OS thread that needs to be linked to the Java thread. Following that, we start it, initiating the execution of the Runnable. Finally, we join it, ensuring that the main thread waits for its completion before proceeding.

Things are straightforward up to this point, but it gets a bit tricky when we want to use the same thread for various tasks. To handle this, we need to set up a task queue and create threads that will wait for items to show up in that queue. This is where Thread Pools come in handy. They offer us this functionality in an efficient and convenient manner right out of the box.

Thread Pools

Thread pools make it easy for us to manage and reuse threads. They come with their own internal scheduling mechanism, allowing us to control both the number of tasks in the queue and the number of threads to maintain.

There are a few flavors of thread pools we can readily use.

Fixed Thread Pool

Fixed Thread Pool

In this thread pool, the number of threads is set upon creation and remains constant throughout its lifespan. This configuration is beneficial when seeking predictability. The worker threads are always ready for use and the maximum number of concurrent tasks remains the same.

int threadCount = 1;
int taskCount = 50;
try (ExecutorService threadPool = Executors.newFixedThreadPool(threadCount)) {
  for (int i = 0; i < taskCount; i++) {
    threadPool.submit(() -> System.out.println("Running task..."));
  }
}

The built-in thread pools adhere to the ExecutorService interface, accessible through the Executors static factory. Initiating a fixed thread pool is straightforward; simply invoke the corresponding method of the factory, namely newFixedThreadPool

Thread pools in Java are autoclosable. This means we can put them in a try-with-resources block, and they’ll wait for all submitted tasks to complete before moving on. However, using try-with-resources will terminate the thread pool after its tasks are complete. If we just want to keep using the thread pool, we can let it live as a field somewhere in the code.

Cached Thread Pool

Cached Thread Pool

The Cached Thread Pool dynamically adjusts its size in response to the workload, prioritizing the reuse of idle threads before resorting to creating new ones. This configuration is useful when we want to adapt allocated resources to accommodate varying task loads. The cached pool doesn’t have an upper limit on the maximum number of threads it can spawn. Consequently, during a substantial influx of tasks, a large number of threads will be created, potentially causing undesired behavior.

Some noteworthy use cases for a Cached Thread Poolinclude:

• Asynchronous processing, particularly when dealing with a varying number of tasks.
• Handling short-lived tasks that sporadically spawn.
• When the requirement is for each task to be executed concurrently.

int taskCount = 50;
try (ExecutorService threadPool = Executors.newCachedThreadPool()) {
  for (int i = 0; i < taskCount; i++) {
    threadPool.submit(() -> System.out.println("Running task..."));
  }
}

The code for creating the cached thread pool is very similar to the fixed one; however, we don’t need to provide the number of threads. Everything else remains the same.

Single Thread Pool

This executor is as straightforward as it sounds — essentially a Fixed Thread Pool with a single thread. Its primary use cases include:

• Executing thread-unsafe code not designed for concurrent environments
• Handling background tasks that lack time sensitivity
• Enforcing sequential execution

Scheduled Thread Pool

Scheduled Thread Pools offer the capability to execute tasks with a delay or at a fixed rate. Similar to the Fixed Thread Pool, it takes the number of threads as a parameter, ensuring that the pool maintains a consistent number of active threads at all times.

The following code will create a new scheduled thread pool with thread size 5 and will submit 50 tasks with 1 to 50 seconds delay each. This code will run for 50 seconds

int threadCount = 5;
int numberOfTasks = 50;
var threadPool = Executors.newScheduledThreadPool(threadCount);
for (int i = 0; i < numberOfTasks; i++) {
  String message = "Scheduled task started with " + i + " seconds delay";
  threadPool.schedule(() -> System.out.println(message), i, TimeUnit.SECONDS);
}

We can use the following code to submit task that will run at a fixed rate of 30 seconds with initial delay of 5 seconds

int threadCount = 3;
long initialDelay = 5;
long interval = 30;
ScheduledExecutorService threadPool = Executors.newScheduledThreadPool(threadCount);
threadPool.scheduleAtFixedRate(
  () -> System.out.println("Every 30 seconds, with initial delay 5 seconds"),
  initialDelay,
  interval,
  TimeUnit.SECONDS);

Custom Thread Pools

ThreadPoolExecutor initialization

All types of thread pools can be implemented using the ThreadPoolExecutor class. It gives us control over every aspect of the pool, including:

• Core Pool Size: The number of threads that are kept alive at all times.
• Max Pool Size: The maximum number of threads the pool can hold.
• Keep Alive Time: The time after which idle threads will be terminated.
• Queue Implementation: By providing our own task queue, we can limit the maximum number of pending tasks or make it infinite. The most commonly used queue is LinkedBlockingQueue, which can be either infinite or have a specified max size. If the max size is reached, new tasks will be rejected.

ThreadPoolExecutor woking when taks a few

The six threads (core pool size) are kept alive at all times, regardless of the number of tasks in the queue. If the queue size never exceeds 6 the pool will never spawn extra threads

ThreadPoolExecutor woking under heavy load

In case of a task influx, the pool will start utilizing the extra space for threads. On top of the 6 threads it already has, an extra 10 are spawned. The total number of threads will never exceed the max pool size (16). The number of new threads spawned is determined by the queue size; in this case, we have more than 16 tasks in the queue, and the thread pool will ramp up to its maximum capacity. For example, if the queue size was 12, only 6 new threads would be spawned.

After tasks have been processed and the queue is again empty, the pool will scale down. It will wait for the specified keep alive time, before removing the surplus threads. The pool is back in its initial state.

Code-wise, the ThreadPoolExecutor is quite straightforward to initialize. After that, it can be used like any other thread pool discussed so far.

int corePoolSize = 6;
int maximumPoolSize = 16;
int keepAlveSeconds = 2;
var threadPool = new ThreadPoolExecutor(
  corePoolSize,
  maximumPoolSize,
  keepAlveSeconds,
  TimeUnit.SECONDS,
  new LinkedBlockingQueue<>());

Conclusion

Thanks a bunch for reading! I really hope you found it helpful. Exciting news — I’m planning to whip up a YouTube video on the same topic, complete with a voiceover and some cool animations featured in this blog post. If you notice I missed something or just want to chat about the topic, drop a comment below. Looking forward to hearing from you!