Scoped Values in Java

To understand the motivation behind Scoped Values, you must have an understanding of Virtual Threads and Structured Concurrency.

Motivation

In the earlier days of Java, when it was necessary to share data across all code executing as part of a single thread, a ThreadLocal instance would be employed. Thread local variables were an effective solution at the time, as the number of threads was limited and the programming style required mutable data. With thread local variables, one could access and modify information throughout a thread’s call chain as needed.

However, the introduction of VirtualThread has significantly changed this landscape. Now, developers are no longer restricted by the number of threads they can create, allowing for potentially millions of threads. While ThreadLocal variables can still be used to access data throughout a thread’s execution and even share that data with child threads created from these threads via InheritableThreadLocal, there may be considerable performance costs when the number of child threads is large, as is permitted by Virtual Threads.

The disadvantages of using ThreadLocal variables with Virtual Threads extend beyond performance issues. ThreadLocal variables are inherently mutable, meaning that any code executing as part of the thread can modify the data held by a ThreadLocal variable. This mutability can make the code more challenging to understand and debug.

Introduction

Scoped Value (JEP-429) is a value that is set once and is then available for reading for a bounded period of execution by a thread. A ScopedValue allows for safely and efficiently sharing data for a bounded period of execution without passing the data as method arguments.

Binding

Consider this example:

private static final ScopedValue<String> USER = ScopedValue.newInstance();

ScopedValue.where(USER, "Tim Nadella", () -> doSomething());

Code executed within doSomething() that invokes USER.get() will read the value “Tim Nadella”. The scoped value is bound while executing doSomething() and becomes unbound when doSomething() completes (normally or with an exception).

ScopedValue defines the where(ScopedValue, Object, Runnable) method to set the value of a ScopedValue for the bounded period of execution. The execution of the methods executed by the Runnable defines a dynamic scope. The scoped value is bound while executing in the dynamic scope, it reverts to being unbound when the run method completes (normally or with an exception). Code executing in the dynamic scope uses the ScopedValue get method to read its value.

Consider this example

public static ScopedValue<String> scopedValue = ScopedValue.newInstance();

public static void main(String[] args) {
  ScopedValueTest instance = new ScopedValueTest();
  ScopedValue.where(scopedValue, "Tim Nadella", () -> {
    System.out.println("Value is: " + scopedValue.get());
    instance.doSomething();
  });
}

public void doSomething() {
  System.out.println("Doing something while accessing scoped value: " + scopedValue.get());
  doSomethingAgain();
}

public void doSomethingAgain() {
  System.out.println("Doing something again while accessing scoped value: " + scopedValue.get());
}

The output of the above example is

Value is: Tim Nadella
Doing something while accessing scoped value: Tim Nadella
Doing something again while accessing scoped value: Tim Nadella

In addition to the where method that executes a run method, ScopedValue defines the where(ScopedValue, Object, Callable) method to execute a method that returns a result.

Inheritance with StructuredTaskScope

ScopedValue become even more powerful feature when used with a StructuredTaskScope.

ScopedValue supports sharing data across threads. This sharing is limited to structured cases where child threads are started and terminate within the bounded period of execution by a parent thread. More specifically, when using a StructuredTaskScope, scoped value bindings are captured when creating a StructuredTaskScope and inherited by all threads started in that scope with the fork method.

Consider this example

interface WeatherService {
  
  String getWeather();
}

class SunnyWeatherService implements WeatherService {
  
  @Override
  public String getWeather() {
    if (WeatherServiceTest.location.isBound()) { //Check whether the value is available
      return "Weather for " + WeatherServiceTest.location.get() + " Sunny";
    }
    throw new RuntimeException("Location not specified");
  }
}

class CloudyWeatherService implements WeatherService {

  @Override
  public String getWeather() {
    if (WeatherServiceTest.location.isBound()) { //Check whether the value is available
      return "Weather for " + WeatherServiceTest.location.get() + " Cloudy";
    }
    throw new RuntimeException("Location not specified");
  }
}

class RainyWeatherService implements WeatherService {

  @Override
  public String getWeather() {
    if (WeatherServiceTest.location.isBound()) { //Check whether the value is available
      return "Weather for " + WeatherServiceTest.location.get() + " Rainy";
    }
    throw new RuntimeException("Location not specified");
  }
}

public class WeatherServiceTest {

  public static ScopedValue<String> location = ScopedValue.newInstance();

  public static void main(String[] args) {
    ScopedValue.where(location, "New York", WeatherServiceTest::getWeather);
  }

  public static void getWeather() {
    try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
      Future<String> res1 = scope.fork(() -> new SunnyWeatherService().getWeather());
      Future<String> res2 = scope.fork(() -> new CloudyWeatherService().getWeather());
      Future<String> res3 = scope.fork(() -> new RainyWeatherService().getWeather());
      scope.join();
      System.out.println(scope.result());
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }

}

In the above example we defined a WeatherService with three implementations, each of the weather service implementations expects the the location to be present inside a ScopedValue.

Now, let’s say to increase the performance & reliability of our application we decided to use all the three WeatherService implementations and return as soon as any one of them returns the weather. This is a perfect case for using StructuredConcurrency where we want to treat the concurrent operations as if they were one.

Using the StructuredTaskScopes ShutdownOnSuccess we start a new scope and call all the weather service implementations concurrently with the fork method, which internally uses VirtualThreads. All the operations forked from the StructuredTaskScope are now able to access the ScopedValue that was set by the parent context.

As a best practice we should always check whether the ScopedValue exists before accessing it using the isBound method.

Scoped values are not bound to a Thread, but only VirtualThreads forked from a StructuredTaskScope.

Rebinding

The ScopedValue API allows a new binding to be established for nested dynamic scopes. This is known as rebinding. A ScopedValue that is bound to some value may be bound to a new value for the bounded execution of some method. The unfolding execution of code executed by that method defines the nested dynamic scope. When the method completes (normally or with an exception), the value of the ScopedValue reverts to its previous value.

Consider this example

public class ScopedValueTest {

  public static ScopedValue<String> scopedValue = ScopedValue.newInstance();

  public static void main(String[] args) {
    ScopedValueTest instance = new ScopedValueTest();
    ScopedValue.where(scopedValue, "Tim Nadella", () -> {
      System.out.println("Value is: " + scopedValue.get());
      instance.doSomething();
    });
  }

  public void doSomething() {
    System.out.println("Doing something while accessing scoped value: " + scopedValue.get());
    ScopedValue.where(scopedValue, "Satya Cook", () -> {
      System.out.println("Value is: " + scopedValue.get());
      doSomethingAgain();
    });
  }

  public void doSomethingAgain() {
    System.out.println("Doing something again while accessing scoped value: " + scopedValue.get());
  }
}

In the above example, when the doSomething method calls that doSomethingAgain method, it does so after re-binding the scopedValue variable to another value, and thus all code executed within that call chain sees the updated value of the ScopedValue variable.

The output of the above code is

Value is: Tim Nadella
Doing something while accessing scoped value: Tim Nadella
Value is: Satya Cook
Doing something again while accessing scoped value: Satya Cook

Limitations

Scoped values are intended for use in relatively small quantities. Initially, the ‘get’ method searches through enclosing scopes to find the innermost binding for a scoped value and caches the result in a small thread-local cache. This makes subsequent ‘get’ invocations for the scoped value considerably faster. However, if a program cyclically uses numerous scoped values, the cache hit rate will be low, resulting in poor performance.

This design ensures quick scoped-value inheritance by StructuredTaskScope threads, essentially requiring no more than copying a pointer. Similarly, leaving a scoped-value binding necessitates minimal effort, such as updating a pointer.

As the scoped-value per-thread cache is limited in size, it is crucial for users to minimize the number of bound scoped values in use. For instance, if multiple values need to be passed this way, it is more efficient to create a record class to store these values and bind a single ScopedValue to an instance of that record.