Java introduces preview virtual threads

OpenJDK’s JEP 425: Virtual Threads (Preview) feature proposal shows that the Java platform will introduce the virtual threads feature. Virtual threads are lightweight threads that can significantly reduce the effort of writing, maintaining, and observing high-throughput concurrent applications.

Java developers have always relied on threads as the building blocks of concurrent server applications, where statements in each method are executed within a thread, and each thread provides a stack to store local variables and coordinate method calls, as well as context trapping when errors are reported. Threads are the concurrent unit of Java and the core foundation of Java tools: the debugger executes statements in threaded methods incrementally, and the parser visualizes the behavior of multiple threads.

Currently, the JDK implements its platform threads as wrappers for operating system (OS) threads, where each instance of the JDK is a platform thread that runs Java code on the underlying OS thread and captures OS threads throughout the life of the code. The number of platform threads is limited by the number of OS threads, which are costly and cannot take up too much. As a result, this current approach to the JDK’s threading implementation limits the throughput of its applications to well below the level supported by the hardware.

About virtual threads

Virtual threads java.lang.Thread are threads that run Java code on the underlying operating system thread (OS thread), but do not capture instances of the OS thread for the entire lifetime of the code. This means that many virtual threads can run Java code on the same OS thread, thus effectively sharing it.

Virtual threads are lightweight implementations of threads provided by the JDK rather than the OS, and are a form of user-mode threads. User mode threads were known as “green threads” in earlier versions of Java , when the concept of OS threads was not mature and popular enough and all green threads in Java shared an OS thread (M:1 scheduling). As the concept of threads evolved, green threads were eventually overtaken by the current platform threads, implemented as wrappers for OS threads (1:1 scheduling), while the newly introduced virtual threads use M:N scheduling, where a large number (M) of virtual threads are scheduled to run on a smaller number (N) of OS threads.

Higher throughput

Developers can choose whether to use virtual threads or platform threads, but virtual threads perform better in high-throughput server applications. For example, the following code that sleeps for one second creates a large number of virtual threads, and the program first gets an ExecutorService, which creates a new virtual thread for each submitted task, then submits 10000 tasks and waits for all of them to complete:.

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 10_000).forEach(i -> {
        executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));
            return i;
        });
    });
} // executor.close() is called implicitly, and waits

Modern hardware can easily support 10,000 virtual threads running such code simultaneously. If the program uses an ExecutorService that creates a new platform thread for each task, such as Executors.newCachedThreadPool(), then it will try to create 10,000 platform threads, which means 10,000 OS threads, and the program will crash on most operating systems. Or maybe the program uses an ExecutorService that gets the platform threads from a pool, like Executors.newFixedThreadPool(200), which is not much better. The ExecutorService will create 200 platform threads to be shared by the 10,000 tasks, which will run sequentially instead of simultaneously, taking the program a long time to run through.

For the above program, a pool with 200 platform threads would only achieve a throughput of 200 tasks per second, while virtual threads would achieve a throughput of about 10,000 tasks per second (after a full warm-up). In addition, if you change 10,000 in the example program to 1,000,000, the program will submit 1,000,000 tasks, create 1,000,000 concurrently running virtual threads, and (after full warm-up) achieve a throughput of about 1,000,000 tasks/second.

In short, virtual threads are not faster threads – they do not run code faster than platform threads. They exist to provide scale (higher throughput), not speed (lower latency).

How do I enable virtual threads?

Virtual threads are currently widely used in other multi-threaded languages (e.g. goroutine in Go and process in Erlang, and are a stable feature in C++), but are still a preview API in Java and disabled by default. To try this feature on JDK XX, you must enable the preview API by

• compiling the program with javac --release XX --enable-preview Main.java and running it with java --enable-preview Main
• When using the source launcher, run the program with java --release XX --enable-preview Main.java
• When using jshell, start with jshell --enable-preview

More information about virtual threads can be found in the OpenJDK’s JDK Issue-8277131, which was currently created on 2021/11/15 and is still in the first phase of the JEP process and is still some time away from a stable release.